project evaluation doc

Evaluation Report February 1 - 5, 2009

Andaman Water Project EMMANUEL HOSPITAL ASSOCIATION

Andaman Water Project, EHA

Evaluation Report – March 2009 1

Table of Content

Background ..........................................................................................................................2

The Evaluation .....................................................................................................................2

Overview of the evaluation..................................................................................................2

Field trips .............................................................................................................................3

Achievements against goals.................................................................................................3

Specific significant achievement of the project. ..................................................................5

Major deviations from the plan and its implications ...........................................................5

Areas where the project could have better addressed ..........................................................5

Key learning from this initiative ..........................................................................................6

Acknowledgements..............................................................................................................7

Conclusion ...........................................................................................................................7

Annexure

Terms of reference – Project End Evaluation......................................................................9

Andaman Water Project - Planned Activities for 2008 – 2009 .........................................11

’First time’ in the Island.....................................................................................................12

Learning of EHA Team from this Project..........................................................................13



Background

Post Tsunami, in the islands of Andaman & Nicobar, Emmanuel Hospital Association (EHA) involved in Tsunami Relief and later followed it with AshaSagar Project which addressed Livelihood among other awareness programs. Subsequently, Andaman Water Project was initiated to address specifically the water needs of the communities identified. A Pilot phase was carried out between April 2007 and March 2008 and followed it with the 2nd year program from April 2008 to March 2009. This entire project is funded by Mennonite Central Committee (MCC). At the end of Pilot phase, certain changes were made to the objectives and technologies adopted.

EHA initiated this evaluation as part of the planned project activity, and this was to be held towards the end of the project. Due to rains and lack of contractors, the construction work has delayed. EHA requested an extension of the project period by two months and MCC accepted this extension. As a corresponding two-month postponement of the evaluation would not have been sufficient to inspect the project in the rainy season, however, the dates for the evaluation were not changed

The evaluators identified were Mr. Richard Heggen, representing MCC and Mr. David Chandran. Together these evaluators have extensive experience in areas related to Water management, Community Development, Project Management, etc.

The Evaluation

This evaluation was carried out from the 1st to the 5th February 2009 and covered the entire project area which is spread over South, Middle and North Andaman.

As set in the Terms of Reference (TOR), (attached as Annexure 1) the purpose of this evaluation is to:

1. To assess the extent of achievement of project goal and objectives in light of the key indicators of change.

2. To find out the appropriateness & effectiveness of the approach in implementing this project.

Overview of the evaluation

The evaluators were able to meet with many of the stakeholders during this period and collect first hand information. We were able to meet with some EHA Volunteers, some of the Partners, and in all the villages we met some of the beneficiaries and the Water Committee members. Details provided at the Presentation in the first day, and the Reports submitted by the project team were also referred to understand the project accomplishments.



A necessary part of project design was that of pilot studies. In a textbook-perfect world, such results would be employed to properly execute the larger execution phase. The duration of this project, however, was relatively short for the multitude of tasks specified. Most of the villagers were out for work. Also we had only brief time at most of the villages. Given the variety of site locations, water technologies and user organizations, questions regarding outcome yet remain at project closure. While a number of technical adjustments were made based on the first-year pilot work, questions remain regarding the efficacy of some of the second-year work.

The EHA project evaluation field trip had time for approximately a dozen of the 30 project sites and in most of these the structures were yet in final stages of construction. In the opinion of the evaluation team, perhaps one-third of these structures could have been improved upon by design review prior to construction, in some cases where unnecessary effort may have been spent (e.g. where a check dam was built higher than its wing walls) and in a few cases where the completed structure may not function as designed. At the same time, the evaluation team recognized the value in leaving most of the location and design particulars to the local owners. Where a technical deficiency is foreseen, however, the evaluation team can only speculate, as there is yet no operational record from which to draw firm conclusions. Adequate follow-up will require a full year's history to observe how the concept of local ownership persists once donation funds are absent and how well the constructed structures function though an annual cycle of rain and drought.

Field trips

Day 1 : Briefing at EHA Office, Port Blair

Day 2 : Visit to villages Rogla Chang, Tikka Dera and Kaushilya Nagar

Day 3 : Visit to villages Narayan Tikery, Sita Nagar, Milan Gram and

Keralapuram.

Day 4 : Visit to Villages Fisherman Colony(Mayabundar), and Lucknow

Day 5 : Visit to Chidya Tapu, Bada Balu and Carbyn Quarry near Port Blair

Achievements against goals

The major Objectives of this project was to:

Objective 1: Improve quantity, quality, and proximity of safe drinking water and reduce incidence of water borne disease in the target population.

Objective 2: Empower local community to maintain and promote the systems

Objective 3: Advocate for improved water supplies to the Island Communities

(Detailed objectives and activities for this year is attached as Annexure 2)



The activities included conducting village survey, having forming Water Committees, deciding on the technology and location in consultation with the local people, construction of Water Structures at the identified location, testing periodically the quality of water, etc.

More number of structures has been constructed than planned and the number of villages covered is 38 against 30 planned. This has been possible due to cost saving by carrying out the construction by direct implementation. The number of families who are covered within the 500 meter distance is generally about 10- 12 families. One structure is not enough for all members of the village to reach within 500 meters.

Quality of water has been tested and in few places not suitable for drinking. What is to be done if the water quality is not acceptable is not clear, and this has to be informed to the village community. Water quantity is sufficient as on date. Whether all of them would provide water during the worst summer months is yet to be seen.

Household water filter units have not yet been supplied. The model tried in the pilot has been dropped. The right product is being assessed. The list of beneficiaries has been made. Training the beneficiaries of its operation and maintenance is necessary

Water committees have been formed from the Self Help Group, which were formed by EHA for other activities. Awareness programmes have been conducted by the volunteers and partners, and some Water Committee members from each village have attended common training programmes. A continued training input is necessary. At present, these members are enthusiastic and committed to the project.

Participation by the community and the partners was visible. Community members, particularly the Water Committee members have shared towards the cost of these structures. This is one of the reasons for the savings in the budget by EHA. This has created an ownership feeling of the community members.

EHA has strategically used local partners in this project. MOU has been signed with the partner’s leadership at the national level. This has ensured that the partner organisation is committed to this project.

Many training programmes have been conducted for partners and representatives from the villages. These have been helpful and have created an interest to promote safe drinking water.

Regarding advocacy efforts, EHA conducted a National Seminar on Water needs in January 2009, and at various occasions their staffs have been representing the need to the government authorities. A Regional Seminar was held in March, 2008. Mapping of existing Bore wells in Andaman has been done and consolidation and analysis is in progress. A video documentary on the water



situation in the Islands has been completed and was released during the National Seminar. Further efforts are expected to continue until the end of this project.

It is recognized and appreciated that the team has managed more challenges than they have anticipated. They have been flexible and persistent. The work done by the staff and volunteers is commendable – considering the distances, hilly and forest terrain, and the rainy season. Significant savings were made on structures due to direct implementation – enabling more beneficiaries to be covered within the available budget.

Specific significant achievement of the project.

More number of structures was constructed than what was planned. Water committees have been formed in all the villages and there is community ownership to this project. Partners are quite excited that safe drinking water is made available to some of the villages where they work.

The EHA Project team has reported many FIRSTs in this project. These are attached as Annexure 3.

Major deviations from the plan and its implications

Timeline for construction has deviated. There are enough reasons to justify the delay.

Since the constructions of the structures have been delayed, the beneficiary education program will need to be continued for both health and water structure maintenance.

Areas where the project could have better addressed

Though we cannot say that the project failed to address in some area, it could be said that there are aspects in which the project could have done better.

For instance, the construction of wells could have been done during the summer months, so that the depth of drilling could be maximized. Some partners who have been selected may not be keen in the future, to address water issues in their field areas. This is due to the nature of the partner organization. More technical input could have been taken before identifying the specific technology and parameters for each location. Or with a midway technical assessment these structures could have been modified according to field observations.

Having taken up a 2 year time-limited project, it would have been better to specify an Exit Strategy and work towards it. However, everyone seems to know that the project will conclude and that they will have to manage on their own after that.



Key learning from this initiative

The model followed here in forming Water committees out of the Self Help Groups (SHG) has been effective and could be replicated elsewhere.

While planning the time-frame, need to consider slow construction periods in the islands where material shortage and distances are real hindrances. The rainy season should be considered when planning the project cycle instead of following a stereo typed cycle of April to March.

The evaluation team recognized the value in leaving most of the location and design particulars to the local owners. The technology water project had to offer in the second phase was simple – open well, recharge structure or hand pump -- and the choice was made in consultation with the community. Building the technical knowledge (good practices) of the local people was achieved.

A crucial adjective in the preceding paragraph, however, is "most." A community may be better in knowing construction methods than what to construct.

The EHA Project team has reported some of their learning from this project. These are attached as Annexure 4.

Recommendations to EHA & MCC

Having assessed the project the evaluators would like to make the following recommendations:

1. 1At the end of the dry season, 2010, all sites should be visited by an evaluator knowledgeable on what was originally planned, both in terms of community involvement and water provision.

1 The inspection and reporting would take one person-month, but that estimate is best left to those

with more field experience. (Note: this proposal does not include intervention or user support during the year before the follow-up. Such a program may or may not be merited, but is different from documenting what happened without continued intervention.) The FOLLOW-UP EVALUATOR could be either another outsider or someone already familiar with the work, but should be someone able to render informed technical opinions. It would be well utilize the present partner/point-persons to support the FOLLOW-UP EVALUATOR, but it is recognized that many of these persons may by then have other duties. The work product would not be simply a score card on the project sites, but rather a set of generalizations useful for other projects in similar areas. What worked and what didn't? In the jargon of engineering, such follow-up is a component of "Total Quality Management." From a common sense point of view, such follow-up flags things that could be done better and/or more efficiently the next time in addition to rectify existing short-comings.



2. Record the soil strata details for all the structures constructed by EHA and share it with the Village Water Committee, local project partner, and Andaman government departments.

3. Other data that could be shared is the Consolidated / Analyzed data on existing Bore wells in the Island of Andaman & Nicobar (as planned).

4. The present EHA team should leave behind a Master Plan for each village and project the number of water sources needed for each village and also the kind of technology to be used.

5. Training programs for use of household water filter systems is required before the project ends.

6. A clear Withdrawal Strategy (as planned) may be developed so that the Learning and Knowledge gained here is disseminated to the appropriate stake holders.

Acknowledgements

The evaluators would like to thank all the EHA staff members, and specially Mr.A.Dennyson, the Project Manager of this project – for all their support and helpfulness. Our thanks to the project partners who took time to share what this partnership has meant to them. Our thanks to volunteers both at the EHA office and at the field level, who took time to be with us in our visits. Our thanks to the beneficiaries, some who had to wait for us, and share their joys with us.

The evaluators would like to thank the leadership of both EHA and MCC who provided us this valuable opportunity to personally participate in their efforts to bring safe drinking water into the homes of the people living in very remote areas.

Conclusion

The Andaman Water Project has addressed one of the very critical needs in the lives of the people who do not have access to safe drinking water, in sufficient quantity and round the year. Though we know that this doesn’t solve the total problem, it has initiated a model by which NGOs such as EHA and its partners can address effectively in certain areas where the government has limitations.

Empowering the village communities and training the local partners is very significant in that it has the potential for a long term advocacy activities.

The impact can be seen only in the long term, and hopefully all will end well. A continued interest in the partners and their activities towards promoting safe drinking water would sustain their interest and keep the advocacy efforts active. However, EHA & MCC can be satisfied that they have initiated and tried their best to address an area which has been neglected for too long.



Annexure – 1

Terms of reference – Project End Evaluation Andaman Water Project

Emmanuel Hospital Association

Project Background: Though Andaman receives good amount of rainfall, about 3000mm annually water supply

during February/March until April/May is rationed and sometimes single connection is

shared between five or six households. During 2007 the supply was as low as 20 minutes

every seven days in Port Blair. The quality of the piped water is also far from ideal: The

water is murkier during rainy season and many islands only raw (untreated) water is

supplied. There are also remote villages, which are not piped; these villages rely on open

wells or surface water. During the rainy season, the primary water source for many

households is a digi, or small temporary well constructed by digging a hole near the

house. Water (a combination of surface runoff and rainwater) is collected from these

holes and is used by the households. Post Tsunami, EHA was already implementing AshaSagar Project phase II through its partners since Jan 2006 funded by Tearfund, UK. This project is working on livelihood, health and disaster preparedness. The partnering agencies are Gossner Evangelical Lutheran Church (GELC), Methodist Church, PILARS Health center, Indian Evangelical Mission (IEM), Shiloh Mission, and Hindi Baptist Church. Given the situation, the drinking water project was started as a time bound initiative funded by MCC, India in July 2007. The target villages are located in North, Middle and South Andaman areas. The water project is expected to end in May 2009. Starting points:

1. Compiled doc on water Project or 1. Project Proposal - General Plan Format 2. Revised Approach 3. Summary of Review Meetings 4. Second year activities

Objectives of this evaluation are:

1. Progress made against the specified Project Plan Objective: To assess the extent of achievement of project goal and objectives in light of the key indicators of change

2. Program Implementation approach Objective: To find out the appropriateness & effectiveness of the approach in implementing this project.

Evaluation Team: 1. Richard Heggen - Team Member Professional Engineer & Hydrologist

Canada 2. David Chandran - Team Leader Former Director Programmes EFICOR



Chennai Mr. Abraham Dennyson, Project Manager will coordinate the logistics at Andamans. Time frame & Plan: It is proposed to conduct the Evaluation from 01 February to 5 February 2009. Date & Time Place Activity

01 Feb Sun (14:30 – 16:30) Project Office, Port Blair Presentation from Project team

02 Feb Mon (04:30 – 09: 30) Roglachang, Baratang Island Travel & visit site, meet user group

02 Feb Mon (09:30 – 14:00) McCarty/ Foster Valley, Middle Andaman

Travel & visit site, meet user group

03 Feb Tue (07:00 - 14:00) Narayan Tikrey / Sreenagar, North Andaman

Travel & visit site, meet user group

03 Feb Tue (14:00 -16:00) Diglipur, North Andaman Meet Partners 04 Feb Wed (05:00 - 12:00) Travel Back to Port Blair 05 Feb Thu (09:00- 11:00) Port Blair Debriefing project team Reporting Schedule: Draft Report - By 20 February 2009 Final Report - By 28 February 2009 Output: A comprehensive evaluation report that would contain

- Specific significant achievements of the project so far

- Major deviations from the plan & its implications

- Areas where the project failed to address

- Key learning from this initiative

- Recommendations to EHA, MCC if any Consultant Fee and other Allowances: The project shall bear the to and fro travel cost from the source to Port Blair and arrange accommodation and travel in Islands. An honorarium of Rs. 5000/- per day shall be payable to the consultant up to 10 days or Rs. 50,000.00 (Preparation 2 days, Evaluation Trip 6 days, Reporting 2 days)



Annexure – 2

Andaman Water Project - Planned Activities for 2008 – 2009

Goal: This project seeks to ensure reliable and safe drinking water supply of the target

villages

10.1 Objectives: What are the Plan’s objectives (how the goal will be achieved)?

1. Improve quantity, quality, and proximity of safe drinking water and reduce incidence of water borne disease in the target population.

2. Empower local community-based groups to maintain and promote the systems

3. Advocate for improved water supply to island communities (outside target area)

10.2 Activities: For each objective, what major activities will be undertaken to achieve it? Activities for 2008-09 Objective 1: Improve quantity, quality, and proximity of safe drinking water and reduce incidence of water borne disease in the target population. Activities:

1. Conduct household survey in 20 villages to determine the above factors so as to determine the extend of problem and decide appropriate technologies to address it.

2. Construct 15 recharge structures (well or pond) with combination of shallow boreholes at safe distance for extraction of drinking water.

3. Construct 2 to check dams or do gully plugging combined with extraction bore wells in the downstream.

4. Construct or rehabilitate 10 ring wells in areas where wells are capable of supplying sufficient fresh water.

5. Provide 500 household water treatment system, consisting of filter and measure for proper dosing of bleaching powder) which are relying on surface water.

6. Construct the above water systems in close proximity so as it is within 500m access

7. Test existing water supplies to determine presence of contaminants and suggest remedial measures for the same.

8. Train community in conservation, safe handling of water, sanitation, and personal hygiene.

Objective 2: Empower local community to maintain and promote the systems

1. Develop 20 village water committees or Small Help Groups to maintain & promote the system

2. Train 60 local personnel in the operation, repair and maintenance of installed systems

3. Train and equip 30 village water committee & volunteers to test drinking water quality and train them to take remedial measures.

Objective 3: Advocate for improved water supplies to the Island Communities (outside target area) 1. Map existing bore wells in the Island and thus make available white paper on

groundwater situation. 2. Conduct 1 regional seminar or 1 national workshop to create awareness and

highlight the water related issue in the Islands among the different stakeholders.

3. Conduct meetings with district and panchayat level officials. 4. Produce a documentary video on the water and water situation in the island. 5. Campaign for improved water supply and responsible water usage through

media.



Annexure – 3

’First time’ in the Island

The EHA Project team has reported on their achievement as ‘First time in the

Island’ the following items:

1. Most of village communities have come together for the first time to work together

in addressing the water issues through the project. Till this time they were dependent on PWD of Panchayat to provide water.

2. 1.5 m diameter was not common in the Islands; Water Project has introduced this size for the first time in the inter Islands. Commonly available sizes are 1 m or 2 m diameter ring wells

3. Hand driven shallow bore wells were first time introduced in the Islands. This way even remote jungle villages can have bore wells

4. For the first time in the Islands custom designed – assembled tanks (22,000 litre) were piloted for water storage. These tanks will be very useful during times of emergencies and relief.

5. For the first time Self Help Group (SHG) has been acting as water user groups in the Islands and found to be quite successful.

6. EHA – MCC had taken first initiatives in water advocacy in the Islands. The administration had been positive so far in giving a hearing to us.



Annexure – 4

Learning of EHA Team from this Project 1. Boreholes in North and Middle Andaman has not been very successful and hence not

recommended as good groundwater aquifers are not found in the Islands except in very few igneous pockets of Mayabunder Island

2. Shallow bore wells though they deliver water in these places the yield has been poor because of the clay sub surface formation and very often fail during summer season.

3. Bigger the diameter better the yield of open dug wells of about 6m depth this is also associated with the clay subsurface formation

4. Dug wells still remain the best source of water for the rural mass of the island however open defecation (lack of sanitary facility) remains a threat of contamination to such shallow open dug wells. (This also the matter of concern in Port Blair as it doest not have a sewer system in the town.)

5. One of the important reasons rainwater harvesting efforts by government has failed is because the dry period and size of storage is not taken into consideration while designing.

6. Though no harmful chemicals found into groundwater some water sources have known to deliver iron rich water or brackish water. In many cases the brackishness is due to the sedimentary rock formations rather than a saline aquifer.

project evaluation doc

Documents