examining early child development: issues relating to measurement lia fernald, ph.d. school of...

Examining Early Child Development:Issues Relating to Measurement

Lia Fernald, Ph.D.

School of Public Health

University of California, Berkeley

March 21, 2013, Universidad de Chile

Desarrollo en la primera infancia: Evidencia longitudinal

In memory: Patrice Lee Engle

1944-2012

Today

• Measuring child development

• Domains of development to be measured

• Theoretical decisions in selecting instruments

• Chile: Early Childhood Longitudinal Survey

Introduction: Why measure child development?

Percentage of disadvantaged children under 5 years old by country in 2004Grantham-McGregor et al., Lancet (2007)

>200 million disadvantaged children worldwide

Differential risk and vulnerability• Children’s development from

0-5 is dependent on quality of early environments and relationship with caregiver.

• Young children growing up in poverty are disproportionately exposed to a wide range of risk factors:

• Poor nutrition• Less stimulating learning

environments• Poor sanitation• Stressful life events• Exposure to environmental

risks

Poverty and cumulative risk

• Number of risk factors increases over time.– Cumulative effect of risk factors

becomes more evident as children get older

• Higher cumulative levels of risk are associated with:– Poorer cognitive development – Psychological distress and

behavioral problems– Slower and lower quality

communicative development

Conceptual framework

From Walker and al. Lancet, 2007

• Environmental factors– Psychosocial risks: harsh disciplinary techniques, maternal depression– Biological risks: malnutrition and infectious diseases

• Poverty and socio-cultural factors increase likelihood of both types of risks

Direct &IndirectEffects

Timeline of development

• Early childhood is characterized by developmental spurts and plateaus

• Skills emerge at different rates and ages

Timing of human brain development, from Grantham-McGregor, et al., 2007

Ecological model of child development

Adapted representation of Bronfenbrenner’s ecological model of child development (Wortham, 2007)

The “Toolkit”• Importance of measuring

child development• Domains of development to

be measured• Theoretical decisions in

selecting instruments• Modification, adaptation and

standardization of existing tests

• Creation of new tests• Training and quality control• Conclusions and

recommendations

Domains of development

• Domains of development:– Cognitive– Language– Motor– Executive function/self-

regulatory– Social/emotional

• Domains are overlapping and mutually influencing

• Every effort should be made to include multiple domains when assessing children’s development

STEP 1: Define purpose of assessment

For example:1. To plan interventions or services; 2. To monitor programs; 3. To conduct impact evaluations;4. To investigate the effect of interventions or programs on specific outcomes of interest; 5. To design a curriculum for a particular child; or 6. To diagnose and assess child progress



ScreeningBrief assessment; identifies children likely to have problems based on cutoffs derived in test population. Does not yield continuous scores. Useful for examples 1-4 above.


AbilitiesDetailed assessment of child’s maximum skill level for age. Provides continuous scores that allow comparisons within and across children/groups. Suitable for all examples above.


DirectDirect Ratings/Reports

Ratings/Reports ObservationObservation DirectDirect Ratings/

Reports

Ratings/Reports

ObservationObservation

Denver (DDST II)

Denver (DDST II)

Ages and Stages

Questionnaires

Ages and Stages

Questionnaires

Naturalistic sample or structured sampling


Bayley Scales IIIWoodcock-Johnson WPPSIStanford-BinetKaufman-ABCExecutive

function tasks


function tasks

MacArthurCommunicative Inventories


Naturalistic sample or structured

sampling (see IEA’s Child

Coding System)



Coding System)

STEP 2: Determine type of assessment

STEP 3: Determine mode of assessment

STEP 4: Determine which assessment to use (examples below)

Constraints to consider: budget; copyright issues; time allocated for assessment; training needs and administrator capacities; test setting; capacity of respondents; language and cultural differences requiring extensive adaptation of assessment; materials required for administration. *Screening test cutoffs must be developed within population.

Step 1: Define purpose of assessment

Key Questions in Selecting Instruments• What are the goals of the assessment/evaluation?• What dimensions of child’s development do you expect to be

affected by the intervention?– What developmental systems are most vulnerable at a given age range? – What are immediate outcomes and longer term outcomes?

• What are the mechanisms at work?– What physiologic processes are influenced by iodine/iron/poverty?

• What are key elements of context that must be considered in selecting the test?– Urban/rural, level of poverty, parent education.

• At what level will effect be measured? – Individual? Household? Population (then consider test such as EDI)?

• How will the sample be selected? – Population sample? Sub-sample?

• What is the analytic plan?– Are norms relevant and/or available? Will a cut-off score be used?











Reports

Ratings/Reports


Denver (DDST II)

Denver (DDST II)

Ages and Stages

Questionnaires

Ages and Stages

Questionnaires




function tasks


function tasks





Coding System)



Coding System)





Step 2: Determine type of assessment

Screening versus ability test

• Screening tests: brief assessments to identify children who are at risk of having development problems– Inexpensive, quick, and relatively easy to administer– Classify children into categories

• Cutoffs used in one population to classify children should not be applied to another population!

• Ability tests: longer tests that assess the maximum skill level for a child at any given age– Continuous scores that can be used to compare

children’s developmental levels with more precision











Reports

Ratings/Reports


Denver (DDST II)

Denver (DDST II)

Ages and Stages

Questionnaires

Ages and Stages

Questionnaires




function tasks


function tasks





Coding System)



Coding System)





Step 3: Determine mode of assessment

Types of assessments: Direct tests• Pros:

– Data are gathered first hand– Data can be less biased than

parental reports– Potentially wider range of

outcomes can be assessed– Many of the “cons” can be

overcome with careful planning and preparation

• Cons: – Young children can be difficult

to test (sleeping, hungry)– Testers need a lot of training

and oversight– Accuracy depends on testing

demands and child must be familiar with parameters (e.g. best v. worst)

Types of assessments: Parent report• Pros

– Easy to administer and require minimal training and instruction

– Often are quick and easy to complete and to score

– Parents can become involved and express concerns

– Often correlate well with direct assessments

– Teachers can be an additional source of information as children get older

• Cons– Parents and teachers may artificially

inflate scores– Parents may not accurately report

abilities– Parents and teachers may have

different interpretations of items in different cultures

Types of assessments: Observation• Pros:

– Highly valid– Measures behavior in an

identified context– Can provide additional or

confirmatory information for other types of assessments

• Cons: – Requires a lot of time and

training– Need to identify if culturally

appropriate– Difficult coding since

observational codes and definitions are not always clearly defined

Types of observation: Naturalistic observation, Sampled observation, Structured situation











Reports

Ratings/Reports


Denver (DDST II)

Denver (DDST II)

Ages and Stages

Questionnaires

Ages and Stages

Questionnaires




function tasks


function tasks





Coding System)



Coding System)





Step 4: Determine which assessment to use

Other constraints to consider

• Budget: Tests can be very expensive (e.g. $1000 for Bayley); administration time is a budget issue, too.

• Copyright issues: Must obtain permission for most tests.• Time allocated for testing: Direct assessment v. parent

rep. • Training: Capacity for administration.• Test setting: Set-up, lighting, noise, observers• Capacity of respondent: Education/knowledge of parent• Language and cultural differences: Words used in testing

materials, approach used for testing (e.g. speedy response)

• Materials: Must be familiar and/or available (e.g. mirror, ball)

Ethical risks and responsibilities• All assessment protocols must

be reviewed and approved by an ethical review board

• Accuracy and validity are extremely important especially if test scores are being used to identify children “with delays”

• Follow-up (e.g. referrals for at-risk children) should be mandatory even in the context of a developing country.

Broad recommendations• Assess characteristics of the child that the intervention is intending to

affect. – Make sure to measure variables that could also be contributing to the

outcomes (e.g. maternal responsiveness, home environment)

• Decide on the type of outcome measure that is appropriate for the evaluation.

• Rely upon multiple measures of children’s development.– Include assessments of executive function and socio-emotional development

• Consider the cultural context and how it may affect children’s development and school readiness – Always work with local collaborators!

• Look for national level tests where possible and use parent/teacher report when possible.

• Begin following children early in life.

Criteria for being recommended

• Psychometrically adequate, valid and reliable;• Balanced in terms of number of items at the lower end

to avoid children with low scores;• Enjoyable for children to take (e.g. interactive, colorful

materials);• Relatively easy to adapt to various cultures;• Easy to use in low-resource settings, e.g. not requiring

much material;• Not too difficult to obtain or too expensive;• Able to be used in a wide age range.

ELPI 2010Area Test Evaluation Proposal

Overall

Development

EEDP (Scale of Psychomotor Development Evaluation)Validity, Reliability and

Norms

BDI (Battelle Developmental Inventory)Validity, Reliability and

Norms

TEPSI (Psychomotor Development Test)Validity, Reliability and

Norms

TVIP / PPVT (Peabody Picture Vocabulary Test)Validity, Reliability and

Norms

Area Test Evaluation Proposal

Cognitive Abilities

Scale WAIS DigitValidity, Reliability and

Norms

Scale WAIS Vocabulary

Validity, Reliability and

Norms

ELPI 2012Area Test Evaluation Proposal

Overall

Development

TADI (Test of Learning and Child Development) Made it by CEDEP

BDI-ST2 (Battelle Developmental Inventory - Screening

Test 2)


Norms

TVIP / PPVT (Peabody Picture Vocabulary Test)Validity, Reliability and

Norms

Executive Function

SDT (Snack Delay Task)Validity, Reliability and

NormsPTT (Pencil Tapping Task)

BDS (Backward Digit Span Task)Validity, Reliability and

NormsHTKS (Head Toes Knees Shoulders Task)

Area Test Evaluation Proposal

Cognitive Abilities

Scale WAIS DigitValidity, Reliability and

Norms

Scale WAIS Vocabulary


Norms

ELPI: Home stimulation (HOME score)

ELPI: Wealth gradients

ELPI (PPVT): gradients over time

Advantages of longitudinal cohort

• Document specific trajectories of development

• Examine shape and timing of emergence of particular skills

• Steepening of gradient at different ages due to individual or household shocks

• Potential for interventions to affect developmental trajectories

Acknowledgements• Funding and support from World Bank for the Toolkit

– Barbara Bruns– Sophie Naudeau– Harold Alderman– Ariel Fitzbein

• External reviewers for the Toolkit– Frances Aboud, McGill University– Santiago Cueto, Catholic University, Peru– Ed Frongillo, University of South Carolina– Jane Kvalsvig, University of Kwa-Zulu Natal, South Africa– Ann Weber, University of California, Berkeley– Paul Wassenich, University of California, Berkeley– Michelle Neuman, The World Bank– Mary Eming Young, The World Bank

• Research assistants– Robin Dean (UC Berkeley), Kallista Bley (UC Berkeley), Melissa Hidrobo (UC

Berkeley), Anna Moore (Cal Poly), Rose Calnin Kagawa• Photo credits for photographs included in presentation

– Lia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber, Tricia Kariger

Thank you!

Contact information:

Lia Fernald

School of Public Health

University of California, Berkeley

[email protected]

(510) 643-9113

mailto:[email protected]

examining early child development: issues relating to measurement lia fernald, ph.d. school of...

Documents