باسحلا ُمهفَأو باتكلا ُأرقَأ · 2018-04-06 · the moe (cisle, ersp) on a...
TRANSCRIPT
أَقرأُ الكتاب وأَفهُم الحسابُمبادرة القراءة والحساب للصفوف المبكرة
Jordan RAMP initiative midline survey
Aarnout Brombacher,
Senior Education Program Specialist, RTI International, [email protected]
2012 National Survey (156 schools)
2013/2014 Intervention Pilot & National Survey
(42 treatment and 110 control schools)
2014/2015 Remedial Pilot
2015 – 2019 Early Grade Reading and Mathematics
Initiative (RAMP)
2
Background to RAMP
• A national EGRA/EGMA survey was conducted at the end of
the 2013–2014 academic year (in May 2014) to measure the
impact of the Intervention Pilot Research Activity.
• Since RAMP is regarded, fundamentally, an
extension/expansion of that activity, the MoE and USAID, felt
that conducting a national baseline study for RAMP at the
start of the initiative in 2015 was not necessary and that
instead the performance of the students in the control group
of the 2014 National Survey would be used to establish the
baseline values for RAMP.
This decision together with the cohort based approach to
implementation has created severe limitations on the
conclusions that can be drawn with regard to the impact of
RAMP 18 months after the start of the in-service training.
Background to RAMP
Background to RAMP
• The 2012 & 2014 National Literacy and Numeracy Surveys clearly
demonstrated that the majority of Jordanian children in the early grades
are not reading with comprehension or doing mathematics with
understanding (application and reasoning).
Reading
18%Mathematics
14%4
Background to RAMP: Reading
• In the case of reading, the surveys found that children were not
developing key foundational literacy skills largely because teachers had
not been specifically trained in teaching early grade reading using an
approach that develops phonemic awareness and provides deliberate
instruction focused on phonics, fluency, vocabulary, and comprehension.
• Early grade reading instruction in Jordan also does not take into account
the important role that reading aloud to children (both at school and at
home) plays.
5
21%
49%
21%
27%
15%
28%
45%
20%22%
8%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Letter Sounds Invented words Oral ReadingFluency
ReadingComprehension
ListeningComprehension
Per
cen
t of
Stu
den
ts
EGRA zero scores by grade
Grade 2
Grade 3
6
Background to RAMP: Reading (2012 Survey)
Background to RAMP: mathematics
• In terms of mathematics, children’s underperformance on the surveys is
explained by the fact that children memorize mathematical facts, rules,
and formulas without understanding.
• In general, early grade mathematics teachers also have low self-
confidence with respect to doing mathematics and lack skills in teaching
early grade mathematics with a focus on understanding, application, and
reasoning.
7
Background to RAMP: Mathematics (2012 Survey)
8
Num IDQuantDisc
MissNum
Add L1 Add L2 Subt L1 Subt L2WordProb
Grade 2 1% 4% 6% 13% 24% 18% 48% 34%
Grade 3 1% 3% 5% 11% 18% 15% 36% 22%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Per
cen
tage
of s
tud
ents
EGMA: Percentage of zero scores by grade
Background to RAMP: mathematics
Background to RAMP: curriculum
• The 2012 classroom observations revealed that daily lesson content
was informed by the page in the textbook for the day. Teachers were
teaching according to a schedule that determined what would be taught
on each day with little regard for whether or not the children were
developmentally ready for the lesson content.
• An analysis of the curriculum in use in 2012, revealed that:
– Despite grade 3 children not performing well on the letter-sounds reading task, this
skill was not being addressed in the curriculum after grade 1.
– In mathematics, the curriculum in grade 2 required students to add and subtract three-
and four-digit numbers and yet one-half of the grade 2 students in the survey could
not correctly subtract 3 from 19.
• The 2012 National Survey revealed that students were not getting
sufficient instruction in foundational reading and mathematics skills—in
foundational skills that research indicates are predictive of future
success in reading and mathematics—with little hope of having this
insufficiency addressed by their teachers or the curriculum that was in
use in 2012. 9
RAMP
• The Early Grade Reading and Mathematics Initiative
(RAMP) is an activity of the Jordanian MoE
• RAMP will apply the methodologies researched in the
intervention pilot, the remedial pilot and other activities of
the MoE (CISLE, ERSP) on a national scale
• The 5-year goal of RAMP is that:
By 2019, the majority of early grade students in Jordanian
public schools will be reading with comprehension and doing
mathematics with understanding
10
Background to RAMP
• RAMP is a five year nationwide initiative of the Ministry of
Education designed to improve the reading and mathematics
skills of students in Kindergarten 2 through G3.
• The initiative:
1. Has developed and distributed improved learning materials to
every K2–G3 classroom in Jordan;
2. Is training teachers, principals, supervisors, and field
directorate and MoE administrators to provide more effective
research-based reading and mathematics instruction;
3. Promotes community participation in reading and
mathematics education; and
4. Supports nationwide adoption of early grade reading and
mathematics policies, standards, curricula, and assessments.11
Background to RAMP: reading
• The Simple View of Reading (Gough and Tunmer, 1986):
• Decoding = fast and accurate reading of familiar and unfamiliar words in
both lists and connected text
• Language comprehension = the ability to derive meaning from spoken
words when they are part of sentences or other discourse
• Reading comprehension = the ability to derive meaning from written
words
12
Reading
comprehension= Decoding ×
Language
comprehension
It is possible to have strong language comprehension
and still be a poor reader if there is difficulty with
decoding.
Cohort Schools Teachers Students
Cohort 1KG2 – Grade 2
623 (25%)2,651
(27%) 90,349 (27%)Grade 3 1,067
Cohort 2KG2 – Grade 2
1,087 (44%)4,509
(48%) 171,531 (51%)Grade 3 1,942
Cohort 3KG2 – Grade 2
749 (30%)2,458
(25%) 71,388 (21%)Grade 3 960
RAMP implementation
RAMP midline survey
• To measure both the impact of the RAMP Initiative so far and the
progress toward the RAMP indicator targets a midline study was
conducted at the end of the 2016–2017 academic year in May 2017.
• Instruments
– Reading: Early Grade Reading Assessment (EGRA)
– Mathematics: Early Grade Mathematics Assessment (EGMA)
14
RAMP midline survey - sampleC
oh
ort
Re
gio
n
Go
vern
ora
te Students
Te
ac
he
rs
Sc
ho
ols
G2 G3
To
tal
Sy
ria
n
Fe
male
Ma
le
Fe
male
Ma
le
1 North Ajloun 101 106 99 94 400 27 40 20
1 North Jerash 100 98 106 95 399 29 39 20
1 Central Zarqa 121 79 117 82 399 40 39 20
1 South Karak 100 103 104 91 398 14 40 20
2 North Irbid 107 93 126 74 400 35 40 20
2 Central Amman 106 84 130 80 400 61 39 20
2 South Tafilah 114 85 106 94 399 3 40 20
2 South Aqaba 107 63 126 103 399 8 37 20
3 North Mafraq 90 103 82 122 397 80 38 20
3 Central Balqa 109 90 121 80 400 6 40 20
3 Central Madaba 105 102 106 83 396 29 40 20
3 South Ma’an 100 104 105 73 382 8 40 20
Total 1,260 1,110 1,328 1,071
4,769 340 472 240 2,370 2,399
Co
ho
rt
Re
gio
n
Go
vern
ora
te Students
Te
ac
he
rs
Sc
ho
ols
G2 G3
To
tal
Sy
ria
n
Fe
male
Ma
le
Fe
male
Ma
le
1 North Ajloun 101 106 99 94 400 27 40 20
1 North Jerash 100 98 106 95 399 29 39 20
1 Central Zarqa 121 79 117 82 399 40 39 20
1 South Karak 100 103 104 91 398 14 40 20
2 North Irbid 107 93 126 74 400 35 40 20
2 Central Amman 106 84 130 80 400 61 39 20
2 South Tafilah 114 85 106 94 399 3 40 20
2 South Aqaba 107 63 126 103 399 8 37 20
3 North Mafraq 90 103 82 122 397 80 38 20
3 Central Balqa 109 90 121 80 400 6 40 20
3 Central Madaba 105 102 106 83 396 29 40 20
3 South Ma’an 100 104 105 73 382 8 40 20
Total 1,260 1,110 1,328 1,071
4,769 340 472 240 2,370 2,399
RAMP midline survey - sample
The 2017 sample of 4,800 students allows for the
calculation of point estimates at national, grade,
gender and governorate level.
The 2014 sample of 2,153 students only allows for
the calculation of point estimates at national, grade
and gender level.
RAMP midline survey – sample
17Grade 2 students Grade 3 students
RAMP midline survey – sample
18Grade 2 students Grade 3 students
79% of the Grade 2 students in the midline survey
have been exposed to RAMP methodologies for
between 2 and 3 semesters
27% of the Grade 3 students in the midline survey
have been exposed to RAMP methodologies for at
most 3 semesters
أَقرأُ الكتاب وأَفهُم الحسابُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: findings
On balance, the RAMP midline survey provides strong
evidence that the RAMP initiative is making a positive impact,
which if supported and sustained will, in the years to come,
help Jordan achieve the early grade reading and mathematics
outcomes that are critical for nation building.
20
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
Syllable sound
fluency (correct syllables per min.) 25.2 31.7 *** ✓
% correct of items attempted 63.8% 72.5% *** ✓
% of students with zero scores 10.8% 4.8% *** ✓
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
Syllable sound
fluency (correct syllables per min.) 25.2 31.7 *** ✓
% correct of items attempted 63.8% 72.5% *** ✓
% of students with zero scores 10.8% 4.8% *** ✓
Invented words
fluency (correct words per min.) 8.9 12.2 *** ✓
% correct of items attempted 39.4% 49.3% *** ✓
% of students with zero scores 30.7% 15.1% *** ✓
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
Syllable sound
fluency (correct syllables per min.) 25.2 31.7 *** ✓
% correct of items attempted 63.8% 72.5% *** ✓
% of students with zero scores 10.8% 4.8% *** ✓
Invented words
fluency (correct words per min.) 8.9 12.2 *** ✓
% correct of items attempted 39.4% 49.3% *** ✓
% of students with zero scores 30.7% 15.1% *** ✓
Oral reading
oral reading fluency (ORF) 26.6 26.4
% correct of items attempted 63.7% 60.4%
% of students with zero scores 9.1% 10.4%
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
Syllable sound
fluency (correct syllables per min.) 25.2 31.7 *** ✓
% correct of items attempted 63.8% 72.5% *** ✓
% of students with zero scores 10.8% 4.8% *** ✓
Invented words
fluency (correct words per min.) 8.9 12.2 *** ✓
% correct of items attempted 39.4% 49.3% *** ✓
% of students with zero scores 30.7% 15.1% *** ✓
Oral reading
oral reading fluency (ORF) 26.6 26.4
% correct of items attempted 63.7% 60.4%
% of students with zero scores 9.1% 10.4%
Reading comprehension
% correct of items attempted 45.3% 49.8% ✓
% correct 33.8% 36.9% ✓
% of students with 80% comp. 17.9% 21.3% ✓
% of students with zero scores 34.1% 30.5% ✓
RAMP midline survey findings: reading
Subtask Measure 2014 2017
Letter sound
fluency (correct letters per min.) 37.0 47.9 *** ✓
% correct of items attempted 64.9% 76.1% *** ✓
% of students with zero scores 21.2% 7.3% *** ✓
Syllable sound
fluency (correct syllables per min.) 25.2 31.7 *** ✓
% correct of items attempted 63.8% 72.5% *** ✓
% of students with zero scores 10.8% 4.8% *** ✓
Invented words
fluency (correct words per min.) 8.9 12.2 *** ✓
% correct of items attempted 39.4% 49.3% *** ✓
% of students with zero scores 30.7% 15.1% *** ✓
Oral reading
oral reading fluency (ORF) 26.6 26.4
% correct of items attempted 63.7% 60.4%
% of students with zero scores 9.1% 10.4%
Reading comprehension
% correct of items attempted 45.3% 49.8% ✓
% correct 33.8% 36.9% ✓
% of students with 80% comp. 17.9% 21.3% ✓
% of students with zero scores 34.1% 30.5% ✓
There is a statistically significant improvement in
performance on the foundational skills for
reading. Reading with comprehension is a
function of performance on foundational skills.
RAMP midline survey findings: reading
Subtask MeasureGrade 2 Grade 3
2014 2017 2014 2017
Letter sound fluency 38.2 47.3 35.7 48.6
Syllable sound fluency 22.1 29.8 28.7 33.8
Invented words
fluency 7.1 10.8 10.9 13.7
% correct 34.5% 48.2% 44.9% 50.4%
Oral reading
oral reading fluency (ORF) 19.1 21.4 35.0 31.7
% correct 56.7% 56.2% 71.6% 64.7
Reading comprehension
% of students with 80% comp.
7.9% 11.4% 29.0% 31.5%
RAMP midline survey findings: reading
Subtask MeasureGrade 2 Grade 3
2014 2017 2014 2017
Letter sound fluency 38.2 47.3 35.7 48.6
Syllable sound fluency 22.1 29.8 28.7 33.8
Invented words
fluency 7.1 10.8 10.9 13.7
% correct 34.5% 48.2% 44.9% 50.4%
Oral reading
oral reading fluency (ORF) 19.1 21.4 35.0 31.7
% correct 56.7% 56.2% 71.6% 64.7
Reading comprehension
% of students with 80% comp.
7.9% 11.4% 29.0% 31.5%
On the foundational reading skills the 2017
Grade 2 students are performing at the same
level that the Grade 3 students did in 2014.
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
Quantity Comparison % correct 78.9% 83.7% *** ✓
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
Quantity Comparison % correct 78.9% 83.7% *** ✓
Addition L1 fluency (correct items per min.) 11.9 12.5 =
Subtraction L1 fluency (correct items per min.) 9.5 10.0 =
Addition and Subtraction L1 % correct 84.0% 86.0% =
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
Quantity Comparison % correct 78.9% 83.7% *** ✓
Addition L1 fluency (correct items per min.) 11.9 12.5 =
Subtraction L1 fluency (correct items per min.) 9.5 10.0 =
Addition and Subtraction L1 % correct 84.0% 86.0% =
Addition and Subtraction L2% correct 41.9% 46.8% ** ✓
% of students with zero scores 13.5% 7.4% *** ✓
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
Quantity Comparison % correct 78.9% 83.7% *** ✓
Addition L1 fluency (correct items per min.) 11.9 12.5 =
Subtraction L1 fluency (correct items per min.) 9.5 10.0 =
Addition and Subtraction L1 % correct 84.0% 86.0% =
Addition and Subtraction L2% correct 41.9% 46.8% ** ✓
% of students with zero scores 13.5% 7.4% *** ✓
Missing Number% correct 60.3% 64.5% * ✓
% of students with zero scores 3.1% 1.7% * ✓
RAMP midline survey findings: mathematics
Subtask Measure 2014 2017
Number Identificationfluency (correct items per min.) 31.8 36.8 *** ✓
% correct of items attempted 88.1% 92.6% *** ✓
Quantity Comparison % correct 78.9% 83.7% *** ✓
Addition L1 fluency (correct items per min.) 11.9 12.5 =
Subtraction L1 fluency (correct items per min.) 9.5 10.0 =
Addition and Subtraction L1 % correct 84.0% 86.0% =
Addition and Subtraction L2% correct 41.9% 46.8% ** ✓
% of students with zero scores 13.5% 7.4% *** ✓
Missing Number% correct 60.3% 64.5% * ✓
% of students with zero scores 3.1% 1.7% * ✓
There is a statistically significant improvement in
performance on the more conceptual
mathematics tasks i.e. doing mathematics with
understanding.
RAMP midline survey findings: mathematics
Subtask MeasureGrade 2 Grade 3
2014 2017 2014 2017
Addition and Subtraction L1 % correct 82.6% 85.0% 85.5% 87.0%
Addition and Subtraction L2 % correct 36.8% 42.5% 47.5% 51.2%
Missing Number % correct 54.3% 60.2% 66.9% 68.9%
RAMP midline survey findings: mathematics
Subtask MeasureGrade 2 Grade 3
2014 2017 2014 2017
Addition and Subtraction L1 % correct 82.6% 85.0% 85.5% 87.0%
Addition and Subtraction L2 % correct 36.8% 42.5% 47.5% 51.2%
Missing Number % correct 54.3% 60.2% 66.9% 68.9%
On the conceptual mathematics tasks the 2017
Grade 2 students are approaching the
performance levels of the 2014 Grade 3
students.
RAMP indicators
Proportion of learners who, by the end
of two grades of primary schooling,
demonstrate reading fluency and
comprehension of grade-level text.
Proportion of students who, by the end
of two grades of primary schooling,
demonstrate that they can do grade-
level mathematics with understanding.
RAMP indicators
Proportion of learners who, by the end
of two grades of primary schooling,
demonstrate reading fluency and
comprehension of grade-level text.
Proportion of students who, by the end
of two grades of primary schooling,
demonstrate that they can do grade-
level mathematics with understanding.
RAMP indicators
Proportion of learners who, by the end
of two grades of primary schooling,
demonstrate reading fluency and
comprehension of grade-level text.
Proportion of students who, by the end
of two grades of primary schooling,
demonstrate that they can do grade-
level mathematics with understanding.
RAMP indicators
Proportion of learners who, by the end
of two grades of primary schooling,
demonstrate reading fluency and
comprehension of grade-level text.
Proportion of students who, by the end
of two grades of primary schooling,
demonstrate that they can do grade-
level mathematics with understanding.
RAMP indicators
Proportion of learners who, by the end of two grades of primary schooling,
demonstrate reading fluency and comprehension of grade-level text.
RAMP indicators
Proportion of students who, by the end of two grades of primary schooling,
demonstrate that they can do grade-level mathematics with understanding.
أَقرأُ الكتاب وأَفهُم الحسابُمبادرة القراءة والحساب للصفوف المبكرة
RAMP midline survey: discussion
RAMP midline survey: discussion
• For reading there is a statistically significant improvement in
performance on all of the foundational skills for reading.
This is encouraging as reading with comprehension is a
function of performance on foundational skills.
– For comprehension there is a statistically significant
narrowing of the gender performance gap!
• For mathematics there is a statistically significant
improvement in performance on the more conceptual
mathematics tasks i.e. doing mathematics with
understanding.
44
RAMP midline survey: discussion
• On the foundational reading skills the 2017 Grade 2
students are performing at the same level that the Grade 3
students did in 2014.
• On the conceptual mathematics tasks the 2017 Grade 2
students are approaching the performance levels of the
2014 Grade 3 students.
45
RAMP midline survey: discussion
• The impact of RAMP is not yet being felt at a statistically
significant level in the Grade 3 performance.
46
RAMP midline survey: discussion
• The RAMP midline survey has generated Governorate level
performance data for both EGRA and EGMA. This is the first
time that this level of data exists in Jordan.
• This data provides the MoE with the data needed to make
decisions about the effective deployment of scarce
resources.
47
On balance, the RAMP midline survey provides powerful
evidence that the RAMP initiative is making a positive impact,
which if supported and sustained will, in the years to come,
help Jordan achieve the early grade reading and mathematics
outcomes that are critical for nation building.
48
49