Introduction to citizen implementation monitoring

tools and methodologies

CEGAA-OSI EE Partners WorkshopMarch 2009

Teresa Guthrie

Why citizen monitoring and advocacy?

• Citizens fund government expenditure through taxes

• Citizens have rights, governments have to deliver on these rights

• Civil society is a component in a transparent and accountable state. CSOs and individual citizens have to participate in decision making and planning processes affecting their lives

• Civil society has to monitor how their public finances are utilised to advance human rights and development and to reduce poverty

Different implementation monitoring and auditing tools

• Social Auditing (e.g. MKSS, India)• Citizen Report Cards (PAC, India)• Public Procurement Monitoring (e.g. PWI,

Philippines)• Physical Verification/ Infrastructural

Development Monitoring (CCAGG, Philippines)

Materials adapted from the source document by Vivek Ramkumar, IBP: “Our money, our responsibility: A citizen’s guide to monitoring government expenditures”

Social Auditing

Introduction

• A social audit is a community driven participatory auditing of government implementation of programmes

• It complements financial audits performed by government agencies– By verifying quality of service provided, performance

of the state, and strengthening the financial audits• It promotes citizen participation in governance issues– Resulting in better transparency and accountability

Differences Between Social & Financial AuditsFinancial Audit Social Audit

Takes place at government level

Delayed process, conducted periodically

Generalised/ departmental auditing

Concerned with financial controls: Was the money spent as planned?

More concerned with operational efficiency**Was the financial management legislation adhered to during spending?

Was there corruption and wastage?

Takes place at community level by citizens

Citizens conduct spot checks of implementation throughout the programme

Issue or programme specific auditing

Concerned with results of spending: What were the outputs and outcomes of spending?

More concerned with spending effectivenessWhat difference did the spending make? How much wastage and corruption possibly took place? Were there tangible benefits for the beneficiaries?

Social Auditing Methodology

Social Auditing: Methodology

• A proper social audit has 7 steps which recognise local citizens as experts – CSOs can only provide importance assistance to the

community undertaking a social audit– Training of citizens on the social auditing process– Assisting with accessing information to conduct the

audit– Assisting in collating and disseminating information to

the whole community– Documenting the social audit findings and following up

with government officials to demand action

• Identify specific programme for auditing• Identify specific state agencies relevant to the audit• Stipulate period of investigation (how many years do

you want to audit?)• Ask questions for information about:– Access to information– How much information is available? Does it cover

the whole audit period?• How much community involvement is expected?

Methodology: 7 Steps of the AuditStep 1: Identify the Scope of the Audit

• What and how much resources are available from the coordinating CSO for this audit?– resources may include office space in the

community, volunteers, transport, etc• What is the relationship between the coordinating

CSO and government officials– Good relationships assist in accessing information

more easily and in seeking urgent corrective action after the audit

Methodology: 7 Steps of the AuditStep 1: Identify the Scope of the Audit

• Which level of government is responsible for the programme to be audited?

• Develop a map that identifies all programme implementers and stakeholders, funding flows

• Coordinating CSO to identify community members who have worked in programme for firsthand information, e.g. volunteers, project managers, etc

Methodology: 7 Steps of the AuditStep 2: Identify and understand the

management structure of the programme

• Coordinating CSO to secure access to relevant documents such as– Accounting documents (cashbooks, wage rolls,

purchase orders, invoices, signed receipts etc)– Technical project records (book of standards,

measurement books, contract specifications, etc)– Managerial records (stock registers, progress reports,

spending certificates, etc)

• Information requests likely to face resistance from local officials

Methodology: 7 Steps of the AuditStep 3: Obtaining information on Programme

under Audit

• Coordinating CSO to sort out with assistance from volunteer citizens collected information and preparing project files

• Creating simplified charts of data basically summarising information obtained

Methodology: 7 Steps of the AuditStep 4: Collating information once received

• Coordinating CSO to make copies of project documents with summaries and share with community members through door to door visits by volunteers

• Organising meetings with residents who have worked or participated somehow in the programme to verify accuracy and truthfulness of detail on the records

• The social audit team or coordinating body should physically verify project site and visit beneficiaries to verify benefits received as reflected on the project documents

• Findings from these exercises should be recorded and filed in relevant project files

• The team should also invite everyone providing information to come testify at the forthcoming public hearing

Methodology: 7 Steps of the AuditStep 5: Distributing information

• The data collection phase should be used to publicise the upcoming public hearings

• Publicise the hearings as widely as possible• Ensure that the hearing location is accessible to all residents• Provide necessities for the hearing: water, sufficient seating

arrangement, sanitation etc• Provide basic location/ venue – extravagant looking venue may

affect participation and attendance• Key roleplayers need to be identified and have to be present at

the public hearing: coordinating team, eminent community members, local media, local government officials (“to receive community feedback and to respond to it”)

Methodology: 7 Steps of the AuditStep 6: Holding public hearing(s)

• Transforming audit findings into a strong and broader advocacy campaign

• Preparing formal report from the hearing(s) and submitting to senior government officials, media and other interest groups

• Developing recommendations for corrective actions and policy changes to improve services

• The coordinating team should put pressure for changes to take place

Methodology: 7 Steps of the AuditStep 7: Follow-up to the hearing

Citizen Report Cards (CRC)- measures of public satisfaction -

• Clarifying scope of the CRC: which public services will be assessed, how findings will be used and shared

• Building a coalition for credibility: determine legitimacy of participating groups (are these local or are these seen as illegitimate outsiders?

• Prepare to answer the questions:“Whose political interest are they serving”? And “Who is funding you?”)

Methodology: 6 Phases of the CRCPhase 1: Identification of Scope, Actors & Purpose

• Identify users to be surveyed – those who have used the service within the past 6 months or year and run focus groups

• Organise focus group meetings with service providers, solicit their views and ask questions emanating from users discussions

• Use information from users and service providers to design an informed questionnaire

• Define the structure and size of the questionnaire• Questionnaire length, interview duration, rating scales etc • Always include a timeframe when asking questions so that

responses apply to that specific period of interest• Pre-test the questionnaire on similar focus groups and make

appropriate modifications • Translations into local language(s) may be necessary

Methodology: 6 Phases of the CRCPhase 2: Designing a questionnaire

• Identify geographical area from which respondents will be drawn

• Determine survey sample size• Choose sample respondents– “proper sampling is no easy task”

Methodology: 6 Phases of the CRCPhase 3: Sampling

• Select and train a team of research assistants/ enumerators– Training on the purpose of the survey, interviewing skills,

etc is compulsory prior to fieldwork

• Perform random checks of interviews to ascertain that collected data is genuine for credibility issues– Enumerators may sometime mis-interpret the respondent,

and record inaccurate data– Some fraudulent enumerators fill information on

questionnaire without any respondent present !

Methodology: 6 Phases of the CRCPhase 4: Execution of survey

• Statistical skills are essential in analysing quantitative data on questionnaires

• Different data analysis / computing tools could be used, such SPSS

• Use findings to compile a report card

Methodology: 6 Phases of the CRCPhase 5: Analysing the data

• Engage relevant officials, and acquire their responses to be incorporated into the final report

• Engage the media, in a press conference or similar medium. Different media could be used to disseminate information to increase coverage

• Facilitate engagement between service providers and users– This may happen in a form of community meetings, discussion, or

event a more effective public hearing

• Present the message– Presentation of final results, both positive and negative, with

conclusions and recommendations for fixing the problems

Methodology: 6 Phases of the CRCPhase 6: Dissemination

Public Procurement Monitoring- the PWI Approach, Philippines -

Introduction• Procurement monitoring involves analysing procurement

documents and holding government accountable for all procurement transactions

• Large amounts of government funds are spent through procurement processes and this is where civil society can make a big difference: by tracking down original transaction documents from the planning phase to the final stage of payment to service providers, and looking for inefficiencies and irregularities

• PWI have developed the so-called Differential Expenditure Efficiency Measurement (DEEM), a tool to measure corruption and inefficiency in public procurement

DEEM • Measures efficiency both in cost and time• Measures wastefulness in public spending• Identifies causes of inefficiency in procurement/

spending• Assumes competition in free market• Measures for competitive bidding & comparable prices• Examines government procurement documents • Enters data into ten forms that collect relevant

information about the procurement

The DEEM Methodology

The DEEM Methodology:• Form I : The first form provides an overview of the transaction, including

information on a cheque issued in payment of an invoice and the corresponding disbursement voucher – Also check: signatories, names, titles, signatures, dates, amounts

• Form II : Description of items procured and summary of information pertaining to that procurement

• Form III : Addressing the purchase request form, check for number, date, requesting department/section/ person, requested items, estimated costs, purpose, authorised signatures etc

• Form IV : Addressing purchase order, check for consistency with information on purchase request form, corresponding disbursement voucher and supplier details

• Form V : Collects information on the invoice, checks for consistency of information in the disbursement voucher and purchase order, authorised signatures and dates.

• Other forms: cover other stages of the procurement process, i.e pre-bidding process, assessment of bid received, inspection reports, annual procurement planning, minutes of procurement meetings etc

The DEEM Methodology: SummariesThe DEEM team then analyses summary sheets to identify inconsistencies and irregularities in the procurement process, asking questions such as:•Is the purchase request form dated after the purchase order form?•Does the purchase order form show a higher cost for a procured item than the bid document does?•Does the payment invoice show a higher amount paid to a vendor than the purchase order does?•Does the purchase order contain a different quantity than the payment invoice does?•Does the delivery date (as recorded in the goods inspection and acceptance form) the same as the date mentioned in the contract or purchase order, and is any delay accounted for?

Physical Verification- the Abra experience –

Concerned Citizens of Abra for Good Government (CCAGG)

Philippines

The CCAGG Approach

• CCAGG monitors government projects in Abra province of Philippines, increasing community participation in development and public works programmes

• Specialty in monitoring infrastructure projects, using local monitors to verify that construction projects are executed according to contract norms and standard in the “blue book”

CCAGG assumptions

• Information of official government documents is not always accurate

• Verification of government information is key to improving accountability

• Public participation has to happen at community level where projects are implemented

• Projects declared ‘complete’ may have never had happened

Methodology for Physical Verification• Community monitors including housewives, students, out-of-

school youth, observe construction projects and report findings to senior or specialist colleagues such as engineers or accountants

• Monitors are equipped with all essentials, measuring tapes, record books, cameras. Amd voice recorders to assist them in monitoring

• CCAGG uses legal framework and other provisions to access information and to monitor projects:– The “Blue Book” – technical reference guide with specifications, or

standards for public works– Approved plans and drawings for the project under investigation – Program of work containing general information on the project under

study – location, source of funding, contract amounts, work schedule, project officer who approved the document etc

Methodology for Physical Verification

• The monitoring team compares information on original project [planning] documents against both the information obtained by community monitors from their physical inspection and the financial and accounting documents and other technical reports submitted by the contractor upon the project completion

• The team looks for evidence of corruption ot poor performance , including the use of sub-standard materials or fraud in contracting procedures (such as rigged contracts or tenders)

• A final report form includes a summary of the project with all specifications, monitors’ findings from physical verification and recommendations for action by relevant government officials

The end, more work back home!!