Impact Study of the

Roving Caregivers ProgrammeResearch Findings 2008

INTRODUCTION

Many high risk families and children have no access to adequate early childhood care and support services. In addition, many Carib-bean countries lack a sustainable continuum of services from pre-natal onwards where caregivers live below the poverty line and are incapable of accessing, without help, the types of supports neces-sary for the growth and development of their children.

Studies continue to show that optimum care for young children can only be realised by mainstreaming good service provision mod-els. From programmatic differences in home-based success to the preventive strength of early parenting interventions, research con-tinues to demonstrate the strength of early learning interventions in the ability to reach the parent/caregiver regularly, in a focused, structured manner which is culturally respectful and cognisant of the socio-economic realities of the home.

The Roving Caregivers programme, RCP has been implemented successfully in Jamaica since 1993. Since 2002, the programme has been extended to four Eastern Caribbean countries: St. Lu-cia, St. Vincent and the Grenadines, Grenada and Dominica. The programme aims ‘to provide early childhood stimulation to children from birth to three years of age who are at risk by focusing on par-ents and children through a home visitation intervention model.’

In particular it aims to enhance parenting knowledge, stimulate good parenting behaviour and change inappropriate child rearing practices, facilitated through regular visits by the home visitors, called Rovers. In designing the research project, the impact of the programme was measured, focusing on changes in parental com-petence, parenting traditions/beliefs and parental styles.

It also takes into account nutritional, disciplinary practices and parenting knowledge as they affect the family and children.

Before further expansion of RCP to additional countries in the region, the Bernard van Leer Foundation (BvLF) has commissioned a comprehensive evaluation of RCP in St. Lucia to provide insights in the programme’s impact and costs. The evaluation consists of a quantitative and qualitative component.

An experimental evaluation of the RCP in Jamaica in 2004, showed that RCP has a substantial impact on the cognitivedevelopment of young children after one year of enrolment. However, there was no evidence of impact in other Caribbean countries or on the lon-ger term cognitive, nutritional or socio-emotional development of young children.

To help bridge this gap in knowledge, the Bernard Van leer Foun-dation commissioned a longitudinal qualitative and quantitative im-pact evaluation of the Roving Caregivers programme in St. Lucia.

Dr. Eleanor Wint (qualitative) and Dr. Wendy Janssens (quantitative) are the researchers.

1

Dr. Eleanor Wint is currently a lecturer of Qualitative Research Meth-

ods in the Social Work programme at the University of Northern British

Columbia (UNBC). She is well known in the Caribbean region as a re-

searcher, and has written joint research text and published articles on

the community development process. Her contributions include such

texts as “Introduction to Social Research: With Applications to the Ca-

ribbean” and “Social Work Reality No. 2: Illustrative Case Studies” Dr.

Wint is conducting the qualitative research for the Roving Caregivers

Programme (RCP) Impact Study.

Dr. Wendy Janssens is a research fellow at the interdisciplinary Am-

sterdam Institute of International Development (AIID) of the VU Univer-

sity in Amsterdam, the Netherlands. She is currently a visiting scholar

at the Institute of Fiscal Studies (University College London). She is

mainly working on impact evaluations of development programmes with

a focus on health, children and education; such as the Early Childhood

Development (ECD) programme in the Caribbean and health care in-

surance and HIV/AIDS treatment in Sub-Saharan Africa. Dr. Janssens

is conducting the quantitative research for the Roving Caregivers Pro-

gramme (RCP) Impact Study.

Dr. Eleanor Wint

Dr. Wendy Janssens

2

METHODOLOGY

The intention was to adopt a holistic approach and in so doing, use a qualitative design that would better assist in understanding inter-actions between caregivers and children as they occur in the home, the community and the society at large, all the while keeping the child and caregiver at the centre of the study. Understanding parent-ing choices rather than simply documenting or recording what par-ents are practicing was the main reason for utilising this approach.

Seven communities in which RCP was introduced provided the population. Seven matched communities (in terms of being below the poverty line, poor social supports and inadequate educational services at the early childhood learning stage) in which there was no program (NRCP), were also selected. . Forty- four families were randomly selected from the total sample chosen for the study, twen-ty- two caregivers from the NRCP and twenty- two from the RCP. The children ranged between 12 and 36 months. As the design was longitudinal, the methodology employed in the 2008 round of data generation builds on the work of previous years. The intention was to move from a purely observational role to one of inclusion and increasing closeness with the participating families.

Some of the methods applied were observation, group de-briefing, in-depth interviews, questionnaires, play group observations. The focus group used was intentionally left until year 2008 so as to mi-nimise impacting on how caregivers might reflect on their personal practices during these sessions. The focus group was structured be-cause the researchers had particular interest in specific topic areas. Participants in the group however were free to engage in the discus-sion in any way they wished, allowing for freedom of interpretation, expression and sequencing of topics. The group sessions allowed for a high level of discussion without persons being singled out.

The challenge was therefore to observe a natural process in a series of controlled environments. In 2008, caregivers were asked to bring their children to a neutral location which was usually a community centre. They were grouped according to whether RCP or NRCP and by the age of the child (up to 18 months, 18- <24 months, 24 months and over). This meant that generally the participants did not know each other and had not met before the group setting.

In each year, communities were observed using a quasi-transect walk during which a number of chance conversations were held. Se-lect families were visited by the researchers so as to renew acquain-tances and confirm reported living conditions. This was intended to add ethnographic quality to this ecological study where people will be seen doing what they do in as natural a setting as possible.

QUALITATIVE IMPACT STUDY SUMMARY

3

FINDINGS

The 44 caregivers together had an average of 2.17 living children. The NRCP group tended to have one child per caregiver while in the RCP group, caregivers tended to have 2-3 living children with an average of 2.70 children.

Caregivers by and large saw their role as ‘challenging but good’. They not only saw themselves as providers, but as protectors, educators, guides, and disciplinarians, roles in which they were constantly learning and growing. They repeatedly spoke about wanting to be accessible and present for their children.

Recognising that their children learn mainly by imitating what is around and copying what they see and hear, not surprisingly, parents wanted to be positive role models for their children. They recognise learning has occurred when they see changes in behaviours and/or hear the child speak of what they have learnt. In 2007 the data revealed a noticeable difference between the NRCP and the RCP groups in that the NRCP group had a greater ability to talk about changes that they were see-ing in the child. This changed in 2008 with the RCP group showing a significant improvement in their ability to express positive feelings about changes noticed.

The structured play sessions proved invaluable in understanding what play as a learning method meant to the parents. Coupling structured observation with focus group discussions gave the researcher the op-portunity to clarify observed behaviours and allowed parents to display accustomed behaviours. While recognising the link between play and learning, communicating, play as a means of remaining healthy and ac-tive, caregivers in both NRCP and RCP (<18 months) groups felt that it should be limited. Spontaneous gender differences were recognised from the play activity in the choice of toys. Boys liked to play with trucks, wheelbarrows and cars. Girls played with dolls and educational toys. Some of the play was imaginative with an emulation of the mother or father activities.

In the 18 - <24 month age group where the children queried situations and were ready to explore, an interesting difference between the RCP and the NRCP groups was noticed. Although in both groups where play was predominantly parallel or independent, NRCP parents seemed more relaxed with their children encouraging the children to use play as an exercise and enjoyment. They were able to leave their children more quickly to their own devices, to play with them when they felt they needed to and to remove themselves from the child’s immediate atten-tion when required. On the other-hand, RCP caregivers returned to ear-lier tendencies of exhibiting more control and emphasising play as an educational activity rather than just having fun.

QUALITATIVE IMPACT STUDY SUMMARY QUALITATIVE IMPACT STUDY SUMMARY

4

5

FINDINGS

The issue of discipline was identified in the 2007 report as an area in

need of examination in the context of the feelings and practices the

parents associated with it. Both NRCP and RCP groups defined dis-

cipline as teaching children morals, life lessons and how the parents

wanted the child to behave. Parents wanted the lessons instilled early

so they would ‘stick with them when they are older’. They expected the

children to have respect for elders, to do what was right, be polite and

mannerly. Many parents expressed strongly the wish not to be embar-

rassed by unruly or rude children. While there were mixed feelings

around the forms of punishment, both groups expressed the desire to

use corporal punishment a lot less, with the RCP caregivers mention-

ing the Rover’s regular visits and the newly learned parenting skills as

the main contributors to this new found position.

As the parent’s sense of competence grows in tandem with their

child’s accomplishments, not only does the self-confidence grow but

also the ability to try changes in their lives. As the parents in the RCP

programme became better able to identify areas of growth and learn-

ing in their children, the more concerned they became with doing their

best to see expectations met. Their willingness to try alternative ways

of punishment, to shy away from excessive beating and corporal pun-

ishment therefore became more obvious, more well articulated, more

conscious in the 2008 round of data collection. These self reflections

from the parents exhibit increased social cognition and an awareness

of their roles and impact on their children’s development.

Parents in both groups felt strongly that their children’s behaviour (at a

later stage) was directly influenced by the quality of parenting provided

during the early stages of development. Consequently, if the child ex-

hibits developmental problems later in life, these problems are consid-

ered a direct result of lax or incompetent parenting practices.

QUALITATIVE IMPACT STUDY SUMMARY

CONCLUSION

Users of the RCP programme have reported an average score of 77 on

the client satisfaction scale, with ranges from 55- 88 (maximum score=91).

This sense of satisfaction was expressed in terms of learning from the pro-

gramme, their overall sense of satisfaction with the programme and the Rov-

ers, as well as satisfaction with their child’s behaviour since being in the pro-

gramme. The study has helped in understanding the ways the programme

triggers significant changes in parenting practices, behaviour and parental

social cognitions amongst parents from the lower socio-economic communi-

ties. Additional changes have occurred in the areas of hygiene, sanitation

practices, nutrition and where possible the use of space.

The study also identified that as the facilitators of the RCP programmes, the

Rovers are the critical connection between the programme and the fami-

lies. Working with the programme has benefited the Rovers in terms of their

gained self -confidence and sense of self as well as their organisational,

communication and presentation skills. They have received basic early child-

hood education training and their stature in the communities has also grown

as a result of their work.

The findings of the current research continue to support the literature on the

positive impact of the ‘home visit’ model of early childhood care interven-

tions. Holistically, the programme benefits directly the growth and knowledge

development of the children, while indirectly that of the caregivers, the ex-

tended family members and the Rovers.

QUALITATIVE IMPACT STUDY SUMMARYQUALITATIVE IMPACT STUDY SUMMARY

6

QUANTITATIVE IMPACT STUDY SUMMARY

METHODOLOGY

The impact evaluation uses a differences-in-differences methodology which estimates changes in child outcomes since baseline (2006) and compares children living in RCP villages with children living in villages without access to RCP. This method compares outcome changes over time between the RCP and the non-RCP children. To measure impact in terms of developmental changes over time, it was essential to have two data points: one before and one after the introduction of the RCP programme. It uses multivariate regression analysis as well as match-ing techniques. In line with current practices, the study is based on an Intention-to-Treat approach. It investigates programme impact on the entire target group: all eligible children living in RCP villages regard-less of their actual participation in the programme. As such, it provides a measure of programme effectiveness in reaching the targeted chil-dren as well as in enhancing their developmental status.

The evaluation was based on a quasi-experimental longitudinal re-search design. The study follows approximately four hundred children over time. Half of the children live in eight ‘treatment’ communities where the programme was introduced after the baseline survey. The remaining children live in seven highly comparable, matched ‘control’ communities where the programme was not introduced.

At yearly intervals, the children’s cognitive, socio-emotional and physi-cal development was assessed at local health centers. In addition, their primary caregiver is interviewed on a wide range of topics that range from household demographics, income, and social support net-works to parenting experiences and disciplining methods.

Cognitive development was measured using the Mullen Scales of Ear-ly Learning. The Mullen’s consists of five subscales: 1) gross motor skills, 2) fine motor skills, 3) visual reception; 4) receptive language; and 5) expressive language. Socio-emotional development was mea-sured using the Vineland Socio-Emotional Early Childhood instrument. The Vineland consists of three subscales: 1) interpersonal relations, 2) leisure and play, and 3) coping skills. Nutritional status is measured based on the child’s height and weight adjusted for sex and age ac-cording to international standards.

To measure parenting behaviour, four different measures were used. In each survey round, the caregivers of the children in the sample were interviewed using a structured questionnaire. In addition to basic modules on household composition, education and wealth, the ques-tionnaire included four sets of questions on parenting: a) daily parent-child interactions, b) disciplining methods, c) feelings of joy or stress related to parenting and d) (in 2008 only) an extensive module on the home environment.

7

QUANTITATIVE IMPACT STUDY SUMMARY

FINDINGS

The RCP shows positive effects on the cognitive development of chil-dren between 6 and 18 months. Fine Motor Skills and the Visual Re-ception scale improved by almost half a standard deviation for the RCP programme group compared to the control group. These scales reflect, amongst others, improved hand-eye coordination.

The study did not find such strong effects of the programme on the cog-nitive development of older children. Also, impact on socio-emotional development may need more time to materialise. Next survey rounds will shed more light on this.

Within the group of programme participants, evidence also suggests that motor skills and language development have increased more for chil-dren who enrolled in RCP at an early age compared to those who were relatively older when they enrolled. Speech development on the other hand, has improved more for children who were older at programme start. However, these results should be interpreted with some caution: they do not take into account that the decision to enroll a child at an earlier or later age may in fact be correlated with the child’s physical and language development.

The findings point to specific risk factors for child development. House-hold poverty is of particular importance in explaining an increasing gap in cognitive as well as socio-emotional development. Second, a low edu-cation level of the primary caregiver has an additional negative effect on cognitive child outcomes. Finally, children born with a low birth weight or with signs of stunting, show significantly slower development of their Gross Motor Skills and Receptive Language development.

RCP has had a significant and positive influence on the likelihood that parents in RCP communities engage in stimulating parent-child interac-tions with their children such as singing songs together. This propensity has increased substantially more in RCP than in non-RCP communi-ties.

Similarly, caregivers in RCP communities are significantly more likely than those in control communities to tell stories to their child. Such stim-ulating interactions are positively related to cognitive development, in particular to language development as well as socio-emotional devel-opment. Thus, there are indications that stimulating interactions have increased for RCP families which in turn is beneficial for the child’s de-velopment.

8

QUANTITATIVE IMPACT STUDY SUMMARY

Cognitive Development and Welfare Baseline: children 0 – 24 months

100

102

104

106

108

110

112

Average 1

(poorest)

2 3 4 5

(richest)

Welfare Quintiles

Co

gn

itiv

e S

um

ma

ry S

co

re

QUANTITATIVE IMPACT STUDY SUMMARY

FINDINGS

Caregivers in RCP communities are also significantly more likely to think that they can influence their child’s future compared to parents in non-RCP communities. Parental aspirations have been found to have a strong positive effect on future child performance in school. This may prove to be one of the important programme effects in the longer run.

CONCLUSION

When the sample of the children at the introduction of the programme

is disaggregated by age, substantial and significant positive effects of

the programme on the cognitive development of children between ages

6 and 18 months at programme start are found.

The disaggregated sample therefore demonstrates that the sooner

the children are placed in the programme, the larger the impact. We

do not find such positive effects for the children between ages 18 and

30 months at programme start. That is, benefits of RCP are most pro-

nounced for children in the younger age brackets.

The study also reveals an increase in stimulating interactions for RCP

families which in turn is beneficial for the child’s development.

Finally, the study highlights a number of risk factors such as poverty,

low levels of caregiver education, and low birth weight that are signifi-

cantly correlated with lower levels of cognitive and/or socio-emotional

development.

9

QUANTITATIVE IMPACT STUDY SUMMARY

Impact on Child Outcomes Findings show RCP demonstrates a significant positive impact on the cognitive development of younger children (6-18 months old at the start of the programme) In addition, RCP shows a significant impact in the following areas for children who enrol at a younger age:

Fine Motor Skills Visual Reception Gross Motor Skills Receptive language Children enrolled at a later age saw a no ticeable increase in expressive language (speech) Poverty and low maternal education are substan tial risk factors for cognitive child outcomes Impact on Parenting Practices Findings Show Low income parents made changes that impact directly on the development of the child such as awareness of the value of good nutrition, use of space, stimulating parent-child interactions and story telling The largest impact was on first time mothers par- ticularly in the area of discipline Parents addressed problems with a much higher degree of talking and words rather than corporal punishment Parents demonstrated new found self-confidence, new knowledge and parenting skills and were better able to describe child’s development Parents recognised the importance of play in the development of the child

Impact on Rovers

Findings show A home-based intervention that is directed primarily at the child and is showing changes in caregivers’ parenting practices The home visitors (called Rovers), have grown in self-esteem and community status Both families in the RCP and other communities want the programme expanded into additional communities It is a programme designed for those living in poverty Conclusions Early Childhood Development is important in the context of economic growth and poverty reduction Poverty and low maternal education are substantial risk factors for child development There is an urgent need for accessible ECD services for children at risk RCP has proven to yield significant benefits for child development Benefits are largest for youngest children Return on investment for ECD projects are large Family counseling services/active referrals need to be an integral part of any early learning home-based strategy

10

RCP IMPACT STUDY FINDINGS 2008

Caribbean Child Support Initiative (CCSI), C/O CARICAD, Weymouth Corporate Centre, Roebuck St. Bridgetown, Barbados. Tel: (246) 427-8535/6 Fax: (246) 436-1709, URL: www.ccsi-info.org