early childhood education study progress report
DESCRIPTION
Slides from the Learning Circle on 12th MarchTRANSCRIPT
Early Childhood Education StudyProgress Report
Juliane Hencke, Caroline Sharp, Tony Bertram, Chris PascalIEA Standing Committee – 5 October 2013
2
Content of Presentation• Study Overview• Study Management• Summary of 1st NRC Meeting• Conceptual Diagram• Sampling Design• Research Questions• Instruments• Study Milestones and Next Steps
3
Study Overview
• ECES is a cross-national study… – Exploring, describing, and analysing provision of early
childhood education (ECE) – Analyzing how ECE contributes to children’s outcomes
• Aims to…– Provide a framework for countries to benchmark their ECE
systems in an international context– Examine the relationships between policy aims, provision,
staffing, parental satisfaction and child outcomes– Investigate the relationships between characteristics of
ECE and children’s competencies
4
What do we provide with ECES?• Country profiles in terms of system characteristics• Descriptives showing what are the characteristics of
ECE in each country on different levels (system, ECE settings, practitioners, parents, children)
• Once we understand the characteristics, what are the relationships between the different levels within and across countries
• What are the outcome differences in terms of different provider types and how do they relate to the outlined policies
• Identify approaches to facilitate these variations
5
What makes ECES unique?• Diversity of countries• Covers different levels of ECE:
intended - implemented - achieved policies• Covers diverse consumer perspectives (settings,
practitioners, parents)• Provide various outcome measures based on an
international framework
• Examines key structural elements from provision to outcome including policy, providers, parents and children
6
Study Management• IEA Data Processing and Research Center (IEA DPC):
Study coordination, practitioner questionnaire, sampling, data processing, analysis
• National Foundation for Educational Research (NFER): Study framework, ECE setting and home questionnaire
• Centre for Research in Early Childhood (CREC): Policy questionnaire and child assessment
• IEA Secretariat: Translation verification, quality monitoring
7
Study Management• Joint Management Committee (JMC)– Regular meetings of international study consortium
• Project Advisory Committee (PAC)– Expert advice on framework refinement, instrument
development and reporting
• National Research Coordinators (NRCs)– Regular meetings to contribute to and to review key
elements of the study
8
Participants at the 1st NRC Meeting
Austria Denmark NorwayBahrain England PolandBelgium (Flemish) Estonia Russian FederationBrazil Germany SloveniaCanada Hong Kong SpainChile Italy ThailandCroatia Lithuania Turkey (did not show up)Czech Republic New Zealand USA (online)
9
Summary of 1st NRC Meeting• Child outcomes are the aspect of greatest interest to
country representatives• Framework needs to be re-organized, so that the
assessment module is the focus of the study • Research questions need to more directed to outcome
measures• There should be a greater focus on pedagogic practice• Interest was expressed in administering the
assessment to younger children (as national option?)
10
Conceptual Diagram
National/ Regional/ Local Policies
ECE Setting• Structural characteristics• Leadership and quality
processes
Practitioners• Practitioner
characteristics • Process and
engagement
Family and Child• Characteristics • Home Learning
Environment
Child Outcomes1. Social and emotional
2. Dispositional and Learning
3. Physical
4. Language and literacy
5. Numeracy
Parental Satisfaction
Phase 1
Phase 2
Sampling Design
ECE Settings of focused children (200)
Practitioners of focused children
(15 or all per setting; about 1,500)
Focus: Children attending center-based education and care in the year before entering ISCED 1
Child Assessment
Children (15 or all per setting;
minimum: 3,000)
Policy Questionnaire (1)
Parents/Carers of sampled children
Ideally: PSU, first sampling
stage
Second sampling
stage
12
ECES Research QuestionsStudy focus is on how ECE contributes to children’s outcomes 1. What variations exist between and within countries in
outcomes for children attending ECE?2. What aspects of ECE settings and education systems are
related to outcomes for children’s early learning and development?
a. System context, policy aims and QA for ECEb. ECE setting characteristics, aims and resourcesc. ECE staff qualifications and CPDd. ECE staff practices in working with childrene. ECE setting leadership and QA systemsf. Communication with and involvement of families
13
ECES Research Questions3. What aspects of children’s personal and social
background and family context are related to children's outcomes?
a. To what extent do these relationships differ within and between countries?
b. Are there particular aspects of the ECE system or setting that mediate these relationships?
14
ECES Research Questions4. What is the relationship between parental satisfaction
with ECE and key features of the setting?1. How is parental satisfaction with ECE related to family and
child characteristics?2. How is parental satisfaction with their child’s ECE setting
related to child outcomes?
5. To what extent do ECE systems differ in their availability, reach and inclusion of children in the period between birth and ISCED 1?
15
Instruments
Phase 1• Policy QuestionnairePhase 2• Assessment module evaluating different aspects of
children’s development and learning– Practitioner ratings (?)– Assessment tasks
• ECES includes comprehensive set of questionnaires– ECE Setting Questionnaire– Practitioner Questionnaire– Home Questionnaire
16
Policy Questionnaire
• To capture evidence on the wider policy context for ECE in each participating country from birth to age of compulsory schooling.
• Completed by NRC (or country designate) to provide robust national level data (qualitative and quantitative) which will address the study research questions.
17
Identification of Key Learning Domains, Sub-Domains and Indicators• Theoretical Grounding: Vygotskian socio-constructivist
approach to learning which accepts children‘s experience and environment shape learning outcomes;
• Empirical Grounding: thorough review of international evidence to identify those early learning outcomes which are asssociated significantly with later achievement and attainment and which are widely acknowledged as important at end of ISCED 0, and which can be assessed reliably, rigorously and efficiently.
Assessment Module
18
Learning Domains, Sub-Domains, Indicators
Social & Emotional
Competence
Dispositional and Learning
Competence
Physical Competence
Language & Literacy
Competence
NumeracyCompetence
a. Sociability
b. Emotional literacy and well being
a. Attitudes and mindsets b. Underpinning learning skills
a. Gross motor skills
b. Fine motor skills
a. Early language skills
b. Early literacy skills
a. Number concepts and operations
b. Spatial concepts
• Social competence• Self-regulation
• Self-concept
• Exploratory drive and curiosity• Involvement/ interest
• Persistence
• Locomotion• Coordination
• Manipulation and handling
• Speaking and communication• Story picture sequencing
• Comprehension
• Simple number computation • Problem solving
• Language of spatial objects
Domains Sub-Domains Indicators
19
• Psychometrically robust and reliable• Inclusive of 5 Learning Domains and able to
discriminate developmental differences• Graduated assessment (possible 3-7 year span, but
probably most 5-6 years)• Ethical, respectful of children’s rights and well being• Interactive, meaningful and attractive items• Easy to train and implement• Adaptable to different cultural contexts and low
resource settings
Assessment Design Considerations
20
Two assessment tools which will be used as a blended assessment approach for all 5 learning domains to ensure reliability and efficiency in the assessment process:
1. Direct Assessment Tasks (DAT) (by trained External Administrator)
2. Practitioner Rating (PR) (by practitioner who is familiar to the children)
Assessment Tools
DATPR
21
Social & Emotional
Dispositional and Learning
Physical
Language & Literacy
Numeracy
Sub-Domain 1a
Sub-Domain 1b
Sub-Domain 2a
Sub-Domain 2b
Sub-Domain 3a
Sub-Domain 3b
Sub-Domain 4a
Sub-Domain 4b
Sub-Domain 5a
Sub-Domain 5b
I 1a1
I 1a2
I 1b1
I 2a1
I 2a2
I 2b1
I 3a1
I 3a2
I 3b1
I 4a1
I 4a2
I 4b1
I 5a1
I 5a2
I 5b1
Assessment Design
22
• DAT administered by trained External Administrator: 30 mins per child
• PR administered by setting practitioner: 15 mins per child
• Clear and standardised instructions for assessment administration for DAT and PR
• 15 sampled children per setting assessed• Assessment undertaken in 2 week period in setting
during a three month window before the end of ISCED 0• IEA QA procedures applied
Administration of Assessment
23
• No ranking of countries but could for example, draw comparisons against countries with similar models of/approaches toECE
• Separate scales for 5 Learning Domains• Linking of outcomes evidence to in country and between
country systemic, structural and process evidence• Country profile using
radar diagrams with country mean scores,for example:
Reporting of Assessment
Social and Emotional Competence
Dispositional and Learning Competence
Physical CompetenceLanguage and Literacy Competence
Numeracy Competence
0
5
10
Country ACountry B
24
ECE Setting Questionnaire• To help gain a deeper understanding of key aspects of ECE
from the perspective of those responsible for leading/managing ECE settings.
• It will be completed by the leaders/managers of the sampled settings (approximately 200 per country).
• It will take approximately 45 minutes to complete and can be administered on paper or online.
• The questions will capture information on setting characteristics and all three study themes.
25
Practitioner Questionnaire• To help gain a deeper understanding of key aspects of
ECE from the perspective of practitioners• Focus on pedagogical action and the quality of the
settings• Target population: All staff directly responsible for the
education and care of children in the last year of ISCED Level 0
• It will take approximately 45 minutes to complete and can be administered on paper or online.
26
Home Questionnaire• The Home Questionnaire will enable the study to report
on the expectations, experiences and views of parents of children currently attending ECE in the year before ISCED 1.
• It will be completed by the parent/primary caregiver of the sampled children (approximately 3000 parents per country).
• It will take approximately 30 minutes to complete and can be administered on paper or online.
• The questions will capture information on family/child characteristics and will focus on all three study themes.
27
Study Publications• Study Framework– Conceptual underpinnings– To be published after Field Trial (Feb 2016)
• International Reports– Results from international data collection summarized in four
thematic reports– Published in 2015, 2017 and 2018
• Technical Report– Information on methods and procedures
• International Database and User Guide– To be released in 2018
28
Study Milestones – Phase 1
• 2013– Jun 10-11: 1st Project Advisory Committee meeting
(Birmingham/ UK)– Jul 29-Aug 1: 1st National Research Coordinator meeting
(Hamburg/ Germany)– Nov/Dec: Pilot Policy Questionnaire
• 2014– Apr/ May: Main Study Policy Questionnaire
• 2015– Feb: Publish Report 1:
Outcomes of Policy Questionnaire
29
Study Milestones – Phase 2
• 2014– Jan 9-10: PAC meeting (Hamburg, Germany)– Feb 24-28: 2nd NRC Meeting (Rome, Italy)– Sep 15-19: 3rd NRC Meeting (TBA)– Nov: Data Management Seminar (Hamburg, Germany)
• 2015– Apr/ May: Field Test
• 2016– Feb: Publish Study Framework– Apr/May: Main Study Northern Hemisphere Countries– Sep/Oct: Main Study Southern Hemisphere Countries
30
Study Milestones – Phase 2
• 2017– Oct: Publish Report 2: Outcomes Questionnaires
• 2018– Feb: Publish Report 3: Assessment Outcomes– Sep: Publish Report 4: Final Report– Nov: Release International Database