early childhood outcomes center new tools in the tool box: what we need in the next generation of...

Early Childhood Outcomes CenterEarly Childhood Outcomes Center

New Tools in the Tool Box: What We Need in the Next

Generation of Early Childhood

Assessments

New Tools in the Tool Box: What We Need in the Next

Generation of Early Childhood

Assessments

Kathy HebbelerECO at SRI International

Presented at the OSEP EC Conference, December 2008


Today’s TopicsToday’s Topics

The new National Academy of Sciences Report: Early Childhood Assessment: Why, What, How

What should the next generation of assessment tools look like?


Reason for the CommitteeReason for the Committee

Controversy over Head Start National Reporting System (NRS)

Congressional mandate “review and provide guidance on

appropriate outcomes and assessments for young children.”

2 key topics: Identification of key outcomes for children 0-5 Quality and purpose of different state-of-the

art techniques


Organization of the ReportOrganization of the Report

Overview of purposes What to assess: Outcomes

And Quality of Environment How to assess:

Technical discussion of validity Issues related to minority, ELL, children

with disabilities Thinking systematically


2 Key Principles2 Key Principles

The purpose of the assessment should guide assessment decisions

Assessment should be conducted within a coherent system of medical, educational, and family support services, that promote optimal development for all children


PurposesPurposes

Determining an individual child’s level of functioning Individual-focused screening Community-focused screening Diagnostic testing Establishing readiness


Purposes (Cont’d)Purposes (Cont’d)

Guiding intervention and instruction Evaluating the performance of a

program or society Program effectiveness Program impact Social benchmarking

Advancing knowledge of child development


Guidelines from Multiple Guidelines from Multiple GroupsGroups

Assessment should benefit children Assessments should meet professional,

legal, ethnical, standards Assessments should be age-appropriate or

developmentally/individually appropriate Parents/family should be involved in the

assessment whenever possible Assessments should be linguistically and

culturally appropriate/responsive


Guidelines from Multiple Guidelines from Multiple Groups (Cont’d)Groups (Cont’d)

Assessments should assess developmentally/educationally significant content

Assessment information should be gathered from familiar contexts (NEGP), realistic settings and situations (NAEYC), or be “authentic” (DEC).

Information should be gathered from multiple sources.

Screening should be linked to follow-up assessment.


Recommendations Related to Recommendations Related to PurposePurpose

Make assessment purpose explicit and public

Assessment strategy must match purpose


[Which] Outcomes[Which] Outcomes

Why this domain? Evidence of consensus around

value Evidence of continuity within

domain over development or links to other domains

Evidence that it is a target for intervention and affected by the environment


Outcome: Domains (NEGP)Outcome: Domains (NEGP)

1. Physical well-being and motor development

2. Socioemotional development3. Approaches to learning4. Language (and emerging

literacy)5. Cognitive skills (including

mathematics)


Classification ChallengesClassification Challenges

Boundaries are artificial Researchers (and test

developers) use different classification schemes


Recommendations related to Recommendations related to Domains and OutcomesDomains and Outcomes

Go beyond the standard domains of language and mathematics

Need support for better measures of socioemotional development

D-4 “For children with disabilities and specials needs, domains-based assessments may need to be replaced or supplemented with more functional approaches”


Validity and ReliabilityValidity and Reliability

Weight of evidence Not whether an assessment is valid

and reliable Use accumulation of evidence to

judge whether the assessment is suitable for the task for which it is intended.

Evidence pertains to specific type of uses.


Assessing All ChildrenAssessing All Children

Much more research needs to be conducted on the validity of tools with minority children, ELL, and children with disabilities.

Children with disabilities Construct irrelevant skills as a threat to validity

Interrelatedness of domains Functional outcomes Universal design Many tools used with children with disabilities

have not been validated for the purposes for which they are being used.


Instrument Selection Instrument Selection RecommendationsRecommendations

Select assessment with acceptable levels of validity

Examine data produced with the assessment for validity for purposes for which the assessment is being used.

Test developers need to collect and make available evidence related to validity for minority, ELL, children with disabilities.

Be very cautious about reaching conclusions about a group not well represented in the validation sample.


Situating Assessment in a Situating Assessment in a SystemSystem

EC System has A goal for children Strategies to achieve this goal Infrastructure supporting the goal

and the strategies Selects assessment to be

compatible with other elements of the system.


Systems RecommendationsSystems Recommendations

EC assessment system must be part of a larger system with strong infrastructure to support early care and education.

Standards Assessment Reporting Professional development Opportunity to learn Inclusion Resources Monitoring and evaluation


Systems RecommendationsSystems Recommendations

Coherence Horizontally coherent – curriculum,

instruction, assessment aligned with early learning standards

Vertically coherent – shared understanding of goals at all levels; consensus about purposes of assessment

Developmentally coherent – taking into account what is known about young children’s skills and understanding develop


Research NeededResearch Needed

New conceptual frameworks Application of technical advances,

e.g., IRT Span birth to 6 or 7 Social emotional measures Capture children’s growth toward

being able to meaningfully participate in variety of everyday setting

Use of technology


Challenges with Current Challenges with Current Assessments for All Assessments for All Children/Children with DisabilitiesChildren/Children with Disabilities

Domains-based When should assessment look at only one

domain? When should it look at the whole child? How functional are current tools?

Varies Need for tools to assess the three outcomes

How authentic? How real? How much is lost with structured requests? And

for which children?



Tool structure E.g., Work Sampling

Need for specificity – sometimes.. E.g., AEPS; High Scope COR

Do we want predictive validity? Universal design principles

DRDP as the exception Adaptations and accommodations



Diagnosis vs. intervention planning purpose Must these be separate tools? Are the things we want to know about children to

prove eligibility so different from what we want to know to develop an intervention plan?

AEPS in JEI, December 2008 Recognition and Response/RTI’s tools for

young children IGDI’s How do these fit with curriculum-based

assessment?

early childhood outcomes center new tools in the tool box: what we need in the next generation of...

Documents

followup assessment

assessment decisionsassessment

generation of assessment

early childhood assessment

young children

appropriate outcomes

identification of key

domainsbased assessments