Citizen Report Cards for Better Service Delivery

Asha Newsum Leadership Forum of the Asia-Pacific Community of Practice on Managing

for Development Results 23 -24 March 2015, ADB

What is a Citizen Report Card (CRC)

• Pioneered by the Public Affairs Centre, Bangalore, India in 1993 as a technique for

evaluating public services in the city by directly assessing citizens’ and users’ feedback

• Goes beyond data collection to being an instrument for exacting public accountability

through extensive media coverage & civil society advocacy that accompanies the process

• CRCs are compiled from data collected during a randomized sample survey of the users

of public services. The responses are aggregated in order to rate the services.

• Can be used across sectors, geographical regions, demographic profiles – income,

gender; Can be applied as a:

Diagnostic tool: Provide citizens and governments with qualitative and quantitative

information about prevailing standards and gaps in service delivery. Also measure the

level of public awareness about citizen’s rights and responsibilities

Accountability tool: Reveals areas where institutions responsible for service

provision have not achieved mandated or expected service standards. Findings can

be used to identify and demand specific improvements in services;

Benchmarking tool: if conducted periodically, can track changes in service quality

over time and reveal improvements or deteriorations in service delivery. CRCs can be

introduced before and after a new program/policy to measure its impact

Information Generated

Access: how many have access – disaggregated by locations, gender, age, socio-economic group

Usage: what extent it is used, where not used and why Quality: how satisfying, useful, relevant is the service Reliability: delivered to stipulated schedules Incidence & types of problems: how often do they experience problems; do

users complain and to whom, is they a good complaint management system; how rapidly is the problem solved

Responsiveness of Service Providers: responding to user service requirements Hidden Costs (unauthorised payments, undue distance, inconvenient delivery

schedules/mechanisms) Willingness to pay Transparency in service provision (norms and standards disclosed)

CRC Process Key Steps

• Understand socio-political context, barriers and determine potential mitigation measures

• Identify key service challenges through Focus Group Discussions (Providers & Users)

• Finalise Scope, Objectives and Outputs

• Agree management and communications approach

• Prepare Survey Tools including Framing of a Scientific Sample

• Conduct Survey – using local NGOs/institutes/firms

• Code, Analyse & Interpret Data

• Rate Services

• Validate findings – through focus group discussions

• Prepare Recommendations and Communicate Results

• Prepare Post-Survey Action Plan

• Continue Periodic Benchmarking and Public Review

Process and Content would be influenced by type of owner and purpose of the CRC.

Design considerations

Takes 6-12 months; Requires considerable human resources; statistical techniques; Costs high compared to many other social accountablity tools.

Critical Success Factors: CRC Process

Entity that conducts the CRC must be:

• Credible and knowledgeable about the services

• Perceived as non-partisan

• Skilled in understanding survey techniques and quantitative analysis

• Experienced in working with multiple constituents

Post CRC

• Sustaining momentum for change: awareness campaigns, service providers making public commitments in relation to service delivery improvements, continuous monitoring by CSOs and media

• Institutionalisation: working with service providers to integrate assessments of service delivery outcomes in the broader performance management system; enhancing capacities of CSOs to monitor service providers’ performance

Critical Success Factors: Context 1. Citizen-state bridging mechanisms for information exchange, dialogue, and negotiation May require new tools for citizen–state interaction, or the reform of existing mechanisms.

2. Ability and willingness of citizens–and their representatives/ CSOs to engage in effective social accountability and demand government and service provider accountability: Capacity development for CSOs and citizens in technical areas and in mobilization, coalition-building, negotiation, and advocacy.

3. Ability & willingness of the state politicians, bureaucrats and service providers to be accountable and responsive to the public and civil society: Transparency and information disclosure, attitudes, skills, and practices favouring listening and constructive engagement with citizens are critical.

4. Presence of a broader enabling environment: Four key areas of an enabling environment include:

• The policy, legal and regulatory environment for civic engagement • The type of political system, how much political freedom is granted, and a tradition of open

pluralistic debate • The economic basis and financial viability of different forms of civic engagement • The values, norms, and social institutions present in a society that support or inhibit open

and pluralistic debate and critical but constructive engagement

Source: World Bank: Social Accountability E-Guide https://saeguide.worldbank.org/critical-success-factors-and-conditions

Factors influencing a successful CRC

Successful CRCs adapt process, content, outputs and outcomes: To local context including political economy factors; and

Have clarity on purpose/objective and owner with responsibility/accountability for taking forward the recommendations after the CRC.

CRCs must be credible and provide reliable and comprehensive representation of citizens' feedback. This requires a methodology that: Involves systematic sampling across all subsections or segments of citizens -

including those who are satisfied as well as the aggrieved - and presents a picture that includes all opinions.

Makes use of advanced techniques of social science research, for selecting samples, designing questionnaires, conducting interviews, and interpreting results.

Outcomes & Lessons

Major Outcomes of CRCs

• Presents strength and weaknesses of public services

• Helps service delivery managers to assess quality of services

• Provides a bridge for civil society to engage service providers in discussion on service delivery performance

• Enables policymakers to set priorities

Lessons Learned

• Ensure leadership and ownership by service providers, policy makers

• Provide joint training for service providers and implementing partners

• Use comparison when appropriate

• Accompany the results with background information about the service

• Continue periodic survey for benchmarking

Developments since 1993

• The CRC is an accepted tool for promoting and strengthening local level citizen participation around the world.

South Asia: Bangladesh, Bhutan, India, Nepal, Pakistan, Sri Lanka; South East Asia: Cambodia, Philippines, Viet Nam, Central Asia: Bosnia and Herzegovina, Kyrgyzstan, Tajikistan, Ukraine, Africa: Kenya, Ethiopia, South Africa, Tanzania, Uganda. Zanzibar, Latin America: Colombia, Mexico, El Salvador; Central Europe: Turkey

• Covers a mix of sectoral, rural, urban and specialised CRCs

• Used by policy makers, service providers, NGOs and citizens groups

• Questions adapted to address service delivery standards

• Used to inform service delivery improvement programs

• Results presented electronically as a dashboard for management (service providers and policy makers)

Developments in CRCs

Widening feedback beyond users/citizens:

• Higher level officials, service delivery implementers, community monitors

• Highlight issues faced by both demand and supply side – a more balanced view

• Helps providers to understand the gaps and work towards plugging them

A combination of tools for better effectiveness

• Rural CRCs – CRCs to push for change at the regional and district levels and Community Score Cards (CSCs) at the village level to push for joint action

• Maternal health project - CRCs for diagnosis and advocacy at the policy level, CSCs for micro-level joint action, budget analysis for pushing for efficiency in spending

Beyond CRCs – What Next?

• Effort to enhance the diagnostic power of CRCs by going deeper into the factors that underline the problems, by using internal data from governments

Twin tracks analysis to supplement the CRC • Selected Expenditure Tracking (SET): To trace the flows of expenditure

associated with the activities of a service or programme • Function Analysis (FA): To trace the chain of functions that are performed in

the course of service delivery or programme implementation

• Using modern ICT systems for data collection, reporting results, benchmarking and communications

• Greater focus on tracking results by measuring impact

Useful Links for Case Studies

• http://gpsaknowledge.org/knowledge-repository/citizen-participation-in-evaluating-health-services-the-latin-american-experience Latin America

• http://www.hsrc.ac.za/uploads/pageContent/5318/Citizen%20Report%20Card.pdf South Africa

• http://www.wsp.org/sites/wsp.org/files/publications/712200751115_Citizens_Report_Card_-_Nairobi.pdf22-05-2013 Citizen's Report Card on Urban Water, Sanitation and Solid Waste Services in Kenya

• http://www.chemonics.com/OurStory/OurNews/Pages/Citizens-Help-Ukraine-Courts-Improve-Performance.aspx# Ukraine

• http://siteresources.worldbank.org/EXTGOVACC/Resources/BIHWeb.pdf Citizen Review of Service Delivery and Local Governance in Bosnia and Herzegovina

• http://www.pacindia.org Public Affairs Centre

• http://www.pacindia.org/partners/public-affairs-foundation Public Affairs Foundation