progress report: eia phase iv studies - eiabd.com · progress report: eia phase iv studies ... and...

1

Progress Report:

EIA Phase IV Studies 6th December 2015

2

CONTENTS

Headlines ................................................................................................................... 3

Programme Activities and Timeline Update ............................................................... 4

Quasi-Experimental Study Sample and Preliminary Data Analysis ............................ 5

Qualitatative Study Sample .................................................................................... 123

Appendix 1: QE study data analysis tables for English Language Proficiency Study ......... 155

Appendix 2: QE study data analysis tables for Classroom Practice Study ............. 277

Appendix 3: Qualitative Study Field Protocol & Guidance Notes ........................... 333

English in Action (2015). Progress Report: EIA Phase IV Research Studies. Technical report, English in Action, Dhaka, Bangladesh. Author: Tom Power, The Open University Acknowledgements We would like to thank the researchers from the Institute of Education and Research (IER), University of Dhaka and also assessors from Trinity College London (www.trinitycollege.com/) for their support in the collection and entry of data for this study.We also acknowledge the EIA RME team for support during the planning, fieldwork, data entry and cleaning. For more information about English in Action, visit: http://www.eiabd.com/ © English in Action, 2015 Published by English in Action (EIA) in Dhaka, Bangladesh. English in Action House 1, Road 80, Gulshan 2 Dhaka-1212, Bangladesh. Phone: 88-02 8822161, 8822234 Fax: 88-02 8822663 Email: [email protected]

mailto:[email protected]

3

Headlines

1. The first round of qualitative fieldwork has been completed and is in the process of being written up. Quantitative pre-test fieldwork and initial data analysis has been completed.

2. Field Operations for Teacher Development and Research Activities are all proceeding in line with the schedule shared in the proposal, July 2015. No significant amendments to the planned timeline are anticipated at this time.

3. Qualitative study field operations had to be carried out under significantly enhanced security protocols, due to recent shootings of foreign nationals. This and the end-of year exams have constrained access to schools,limiting those available for sampling and significantly increasing the time required for finalising fieldwork logistics.None-the-less, the first round of fieldwork has been completed on schedule.

4. The actual quantitative pre-test sample achieved was very close to the planned

sample. Comparison of control and treatment groups show no statistically significant differences on any major indicator overall. There are some minor statistically significant differences when samples are broken down further (e.g. to divisional level; with secondary male students, but not female or overall secondary students), but these are probably of little educational significance.

5. Pre-test descriptive statistics show English Language Proficiency levels similar to the

2009-2010 baseline.

6. Pre-test descriptive statistics on Classroom Practice show high levels of student talk-time and English language use by teachers and students; however, these do not seem to be associated withhigh level learning outcomes (see 4). Teacher talk is high (54% primary; 62% secondary), with presenting being the most common talk-type. There are low-levels of student-student talk (e.g. pair or group talk) and students spend only around a third of the lessonpracticing the four skills (35% in primary; 28% in secondary) other than listening to teacher.

7. We are considering whether the second round of the qualitative study fieldwork might

be enhanced to allow more insight into the quality of classroom practices, as well as the English-language and socio-economic contexts, of the qualitative study schools. These would be in addition the primary focus on the experiences of support in school.

4

Programme Activities and Timeline Update The implementation of field operations for the teacher development programme and associated research activities (quantitative and qualitative) have occurred in-line with timeline submitted in the proposal (July 2015) (see fig 1below). Pre-test fieldwork was completed by 31’ July, as per budget milestone. Data entry, cleaning and analysis took longer than anticipated, with initial analysis of pre-test data being completed by the end of November, 2015. Qualitative fieldwork was completed as planned, in October and November 2015. Fig1.The planned timeline

A spate of shootings of foreign nationals in Bangladesh in the fourth quarter of 2015 meant field operations for the qualitative study had to take place under enhanced security protocols, which severely constrained freedom of movement, including some international staff having to travel under armed police guard during fieldwork (see fig 2). Security and travel restrictions severely constrained the schools that could be considered for the qualitative sample and absorbed substantial time and human capacity in planning.

Figure 2. Armed Police Escort for Field Visit to Schools, Qualitative Study, November 2015.

Remaining teacher development activities, qualitative fieldwork and quantitative post-test field activities are anticipated to take place as per the timeline, without significant revisions.

5

Quasi-Experimental Study Sample and Preliminary Data Analysis Sample

The sample design was to sample some 240 teachers and 2,400 of their students, from 120 schools, with the same teachers and students taking part in both pre- and post- test. To compensate for student attrition, it was planned to test an additional 1,200 students at pre-test.

Fig. 2 The planned sample

Pre-intervention Post-intervention

Primary Schools Teachers Students Schools Teachers Students

Intervention group

30 60 900 30 60 600+

Control group 30 60 900 30 60 600+

Secondary

Intervention group

30 60 900 30 60 600+

Control group 30 60 900 30 60 600+

Total 120 240 3,600 120 240 2,400+ Fig 3. The actual pre-test sample achieved1

Actual Pre-test Sample

Primary Schools Teachers Students

Intervention group

30 59 888

Control group 30 59 880

Secondary

Intervention group

30 59 892

Control group 27 54 823

Total 117 231 3, 483

PreliminaryData Analysis

Task 1: Statistical comparison of control and treatment groups The primary statistical analysis was to compare the control and treatment groups, checking for any statistically significant differences between them. Detailed tables are provided in Appendix 1.

English Language Proficiency: There were no statistically significant differences (P<0.05) found between control and treatment groups, for:

Primary Students

Primary Student by gender

Primary Teachers

Primary Teachers by gender

Secondary Students

Female Secondary students

Secondary School Teachers

1 From initial data-analysis after cleaning.

6

Secondary School Teachers by gender There are small but statistically significant differences (P<0.05) between control and treatment groups English language proficiency, when considering:

Male secondary students (but not female, or secondary students overall). The pass rate is lower for male secondary students in the treatment group (75%) compared to control group (80%) (See Table 7. Secondary School Students ELP disaggregated by Gender, Appendix 1)

Individual divisions apart from Dhaka (primary & secondary), Syhlet (Primary) and Barishal (secondary) all show small but statistically significant differences between control and treatment groups for students, although there are no statistically significant differences over-all2. (See Table 3. Primary Students ELP disaggregated

by Division and Table 8. Secondary Students ELP disaggregated by Division, Appendix 1).

For teachers’ ELP, the sample size is too small when disaggregated by division, to allow for tests of significant differences.

Classroom Practices There were no statistically significant differences (P<0.05) between control and treatment groups in either primary or secondary classrooms, for:

Proportions of Teacher Talk in English / Bangla

Proportions of Student Talk in English / Bangla

Proportions of type of activity in classrooms (Teacher Talk, Student talk, Student Reading, Writing, Listening; Other activity).

Types of teacher talk (Presenting, Organising, Giving Feedback, Asking Questions), or the proportion of English / Bangla used within most types of teacher talk.

Most types of student talk (Single, Pair, Group, Chorus), or the proportion of English / Bangla used within each type of teacher talk. (Instances of pair and group work were very few for statistical purposes).

In primary classrooms, there were no statistically significant differences identified between control and treatment groups on any indicator. In secondary classrooms, there were some statistically significant differences (P<0.05) between control and treatment group classroom practices:

Treatment group secondary teachers used more English when giving feedback and asking questions, compared to control group secondary teachers (See Appendix 2, Table 10). (The proportion of these types of teacher talk was not statistically different (Table 9), but the balance of language used in these two types of teacher talk was statistically significant. However, there was no difference in the overall balance of talk in English / Bangla by secondary teachers in control or treatment groups (Table 8), just in the use of English and Bangla for these two types of talk).

Secondary student’s participation in choral speaking was higher in the control group (33%) than the treatment group (18%).(See Appendix 2, Table 11).

2 Previous EIA studies have often shown different patterns from one division to another… the UNICEF deprivation study found

deprivation to be relatively evenly distributed at divisional level, but increasingly varied at district and upazila level and below. This might suggest that the smaller sample sizes at divisional levels are prone to sampling effects that average out at the national level. See UNICEF (2010). A case for geographic targeting and basic social service to mitigate inequalities in Bangladesh. Technical report.

7

Summary of Statistical Analysis comparing Control and Treatment Pre-test data. The two groups are very well matched; for almost all indicators, there are no statistically significant differences between the two groups. However, when sample sizes are reduced, for example when looking only at particular divisions, some statistically significant differences emerge.

Task 2: Descriptive Statistics Preliminary draft analysis of descriptive statistics is provided below.

English Language Proficiency

Figure 4. Primary Students GESE Scores

Over half of the pre-test primary students did not achieve level 1 or above on the GESE scale. A third of pre-test primary students achieved level 1, and one-in-ten students achieved a level 2 or 3. A very small number of students (less than half a percent) achieved levels 4 or 5.

Figure 5. Secondary Students GESE Scores

Over a quarter of secondary students did not achieve level 1 or above on the GESE scale. Almost a third achieved level one, and a further third achieved levels 2-4. Few (around 1-in-20) students achieved above level 4.

57

33

6 40 0

0

10

20

30

40

50

60

0 1 2 3 4 5

Per

cen

tage

of

Stu

den

ts a

chie

vin

g gr

ade

GESE Grade

Primary Student GESE scores

28%35%

15%11%

5%2% 2% 1% 1% 0%

0%

10%

20%

30%

40%

0 1 2 3 4 5 6 7 8 9

Per

cen

tage

of

stu

den

ts

ach

ievi

ng

grad

e

GESE Grades

Secondary Student GESE Scores

8

Figure 6: Primary Teachers GESE Scores

Almost two-thirds of primary teachers achieved grades 1-3; a third achieved grades 4-6; very few (a little over one-in-twenty) achieved levels 8 or 9. A very few (one-in-twenty) failed to achieve grade 1. Figure 7. Secondary Teachers

Over a third of secondary teachers achieved grades 1-3. Almost half of secondary teachers achieved grades 4-6, and a tenth achieved grades 7-8. Very few (one-in-twenty) achieved grade 9 or 10.

2%

21%

15%

23%

12% 11% 10%

4%2%

0%

5%

10%

15%

20%

25%

0 1 2 3 4 5 6 7 8Per

cen

tage

of

teac

her

s ac

hie

vin

g gr

ade

GESE Grade

Primary Teacher GESE Scores

1%3%

8%

25%

18%15% 15%

7%

3% 2% 3%

0%

5%

10%

15%

20%

25%

30%

0 1 2 3 4 5 6 7 8 9 10

Per

cen

tage

of

teac

her

s ac

hie

vin

g gr

ade

GESE Grades

Secondary Teacher GESE Scores

9

Classroom Practices: Primary

Figure 8: Primary Students’ Use of English and Bangla

Figure 9: Primary Teachers’ Use of English & Bangla

Figure 10: Primary Classroom Activity

13%

87%

Primary Student Talk

Bangla English

30%

70%

Primary Teacher Talk

Bangla English

53%

23%

5%

5%

1%

13%

Primary Classroom Activity

teacher talk student talk ss reading ss writing ss listening other

10

Figure 11: Primary Teacher Talk by Type

Figure 12: Primary Student Talk by Type

43

23

17

17

Primary Teacher Talk Types

presenting organising feedback questions

40

46

50

Primary student talk types

individual pair group choral

11

Classroom Practices: Secondary

Figure 12: Secondary Students’ Use of English and Bangla

Figure 13: Secondary Teachers’ Use of English & Bangla

Figure 14: Secondary Classroom Activity

22%

78%

Secondary Student Talk

Bangla English

38%

62%

Secondary Teacher Talk

Bangla English

62%18%

3%

3%

3%

11%

Secondary Classroom Activity

teacher talk student talk ss reading ss writing ss listening other

12

Figure 15:Secondary Teacher Talk by Type

Figure 16: Secondary Student Talk by Type

53

15

12

20

Secondary Teacher Talk Types

presenting organising feedback questions

60

4.5

9.5

26

Secondary Student Talk Types

individual pair group choral

13

Qualitative Study Sample

Planned Sample 1. Total: 8 schools: 16 teachers & 8 head teachers. 2. 4 Control schools, 4 Treatment schools 3. Each phase: 2 'high' and 2 'low' performing schools

High performing ~ 75th percentile (+/- 5 points) for student talk and student learning outcomes

Low performing ~ 25th percentile (+/- 5 points) for student talk and student learning outcomes

School Pre-test performance

High Low

Experimental study

control 1 primary; 1 secondary

1 primary; 1 secondary

treatment 1 primary; 1 secondary

1 primary; 1 secondary

Design for Achieving Sample For student talk, we decided to count ‘% of lesson time with student talk in English’ as a variable representing both our key student talk indicators: student talk-time and % of student talk in English. Student talk in English was calculated for classes and then schools, to give a ‘classroom practice’ (CP) indicator for the school. Schools were then ranked by this indicator, and assigned a percentile position. Student GESE scores were calculated for classes and then schools, and schools were then ranked by this school GESE indicator, and assigned a percentile position. Our ideal sample would have been to find identify schools such that:

High performing = 70th-80th percentile in both CP and GESE indicators

Low performing = 20th-30th percentile, in both CP and GESE However, the percentile ranges had to be widened substantially, as we also had to identify at least two schools that were sufficiently co-located to allow a field team to visit them on consecutive days. The sampling was further constrained by limited access to schools during the revision and examinations period, and by travel limitations imposed upon the team as a result of enhanced security protocols due to recent shootings of foreign nationals.Although we had intended to select based upon GESE and CP indicators, where the practical constraints prevented this, we have privileged the selection of the student learning outcomes indicator (GESE) over the classroom practice indicator (CP), in selecting schools for the study.

14

Actual Sample

Phase Performance (GESE& CP)

Treatment (EIA) School

Control (Non-EIA) School

Primary Hi Hathazari-CTG South BuircharAlamia G.P.S 01913630564 GESE: 81st Percentile CP: 72nd Percentile

Hathazari-CTG North Buirchar Rashida G.P.S 01554326046 GESE: 72nd Percentile CP: 19thPercentile

Lo Khulna Binapani GPS 01966017427 GESE: 44th Percentile CP: 14th Percentile

Khulna Krishi College PS 01199340814 GESE: 4th Percentile CP: 42nd Percentile

Secondary Hi Rajshahi Govt PN Girls High School, GESE: 98th Percentile CP: 95th Percentile

Rajshahi Satellite Town High, GESE: 62nd Percentile CP: 22nd Percentile

Lo Khulna Shibrampur High School, GESE: 9th Percentile CP: 36th Percentile

Dhaka Algi High School, GESE: 25th Percentile CP: 4th Percentile

15

Appendix 1: QE study data analysis tables for English Language Proficiency Study Data Analysis by Dr.Nai Li, the Open University.

Primary Schools: Comparison of Control & Treatment Groups

Table 1. Primary School Students ELP

Group Treatment or Control group * GESEgrade

GESEgrade

Total 0 1 2 3 4 5

Group Treatment or Control group

1.00 Treatment group

486 316 42 36 4 4 888

54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%

2.00 Control group

514 269 63 28 3 3 880

58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%

Total 1000 585 105 64 7 7 1768

56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%

There is no significant difference between the treatment and control group. P=.075.

16

Table 2. Primary School Students ELP disaggregated by Gender

Group Treatment or Control group * GESEgrade * Gender

Gender

GESEgrade

Total 0 1 2 3 4 5

Female Group Treatment or Control group


273 173 26 17 1 1 491

55.6% 35.2% 5.3% 3.5% .2% .2% 100.0%

2.00 Control group

285 149 38 16 0 0 488

58.4% 30.5% 7.8% 3.3% 0.0% 0.0% 100.0%

Total 558 322 64 33 1 1 979

57.0% 32.9% 6.5% 3.4% .1% .1% 100.0%

Male Group Treatment or Control group


213 143 16 19 3 3 397

53.7% 36.0% 4.0% 4.8% .8% .8% 100.0%

2.00 Control group

229 120 25 12 3 3 392

58.4% 30.6% 6.4% 3.1% .8% .8% 100.0%

Total 442 263 41 31 6 6 789

56.0% 33.3% 5.2% 3.9% .8% .8% 100.0%

Total Group Treatment or Control group


486 316 42 36 4 4 888

54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%

2.00 Control group

514 269 63 28 3 3 880

58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%

Total 1000 585 105 64 7 7 1768

56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%

There is no significant difference in term of gender difference between the treatment and control group. P=.276; P=.295

17

Table 3. Primary Students ELP disaggregated by Division

Group Treatment or Control group * GESEgrade * Division Crosstabulation

Division

GESEgrade

Total 0 1 2 3 4 5

Barishal Group Treatment or Control group


28 27 1 56

50.0% 48.2% 1.8% 100.0%

2.00 Control group

47 5 0 52

90.4% 9.6% 0.0% 100.0%

Total 75 32 1 108

69.4% 29.6% .9% 100.0%

Chittagong Group Treatment or Control group


46 95 7 1 149

30.9% 63.8% 4.7% .7% 100.0%

2.00 Control group

70 79 1 0 150

46.7% 52.7% .7% 0.0% 100.0%

Total 116 174 8 1 299

38.8% 58.2% 2.7% .3% 100.0%

Dhaka Group Treatment or Control group


120 71 18 25 2 3 239

50.2% 29.7% 7.5% 10.5% .8% 1.3% 100.0%

2.00 Control group

111 66 40 16 2 2 237

46.8% 27.8% 16.9% 6.8% .8% .8% 100.0%

Total 231 137 58 41 4 5 476

48.5% 28.8% 12.2% 8.6% .8% 1.1% 100.0%

Khulna Group Treatment or Control group


96 24 120

80.0% 20.0% 100.0%

2.00 Control group

108 5 113

95.6% 4.4% 100.0%

Total 204 29 233

87.6% 12.4% 100.0%

18

Rajshahi Group Treatment or Control group


50 53 9 6 2 120

41.7% 44.2% 7.5% 5.0% 1.7% 100.0%

2.00 Control group

25 64 20 10 1 120

20.8% 53.3% 16.7% 8.3% .8% 100.0%

Total 75 117 29 16 3 240

31.3% 48.8% 12.1% 6.7% 1.3% 100.0%

Rangpur Group Treatment or Control group


71 38 7 4 1 121

58.7% 31.4% 5.8% 3.3% .8% 100.0%

2.00 Control group

85 36 0 1 0 122

69.7% 29.5% 0.0% .8% 0.0% 100.0%

Total 156 74 7 5 1 243

64.2% 30.5% 2.9% 2.1% .4% 100.0%

Sylhet Group Treatment or Control group


75 8 0 0 0 83

90.4% 9.6% 0.0% 0.0% 0.0% 100.0%

2.00 Control group

68 14 2 1 1 86

79.1% 16.3% 2.3% 1.2% 1.2% 100.0%

Total 143 22 2 1 1 169

84.6% 13.0% 1.2% .6% .6% 100.0%



486 316 42 36 4 4 888

54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%

2.00 Control group

514 269 63 28 3 3 880

58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%

Total 1000 585 105 64 7 7 1768

56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%

There are significant differences between the treatment and control groups in most of divisions, except Dhaka, Sylhet

19

Table 4. Primary Teachers’ ELP

Group Treatment or Control group * GESEGrade012

GESEGrade012 Total

0 1 2 3 4 5 6 7 8



0 10 8 18 7 7 8 1 0 59

0.0% 16.9% 13.6% 30.5% 11.9% 11.9% 13.6% 1.7% 0.0% 100.0%

2.00 Control group

2 15 10 9 7 6 4 4 2 59

3.4% 25.4% 16.9% 15.3% 11.9% 10.2% 6.8% 6.8% 3.4% 100.0%

Total 2 25 18 27 14 13 12 5 2 118

1.7% 21.2% 15.3% 22.9% 11.9% 11.0% 10.2% 4.2% 1.7% 100.0%

There is no significant difference between the two groups.P=.178

20

Table 5. Primary Teachers’ ELP disaggregated by Gender

Group Treatment or Control group * GESEGrade012 * Gender

Gender

GESEGrade012

Total 0 1 2 3 4 5 6 7 8



0 6 6 9 5 4 5 0 0 35

0.0% 17.1% 17.1% 25.7% 14.3% 11.4% 14.3% 0.0% 0.0% 100.0%

2.00 Control group

1 13 8 6 4 5 1 1 1 40

2.5% 32.5% 20.0% 15.0% 10.0% 12.5% 2.5% 2.5% 2.5% 100.0%

Total 1 19 14 15 9 9 6 1 1 75

1.3% 25.3% 18.7% 20.0% 12.0% 12.0% 8.0% 1.3% 1.3% 100.0%



0 4 2 9 2 3 3 1 0 24

0.0% 16.7% 8.3% 37.5% 8.3% 12.5% 12.5% 4.2% 0.0% 100.0%

2.00 Control group

1 2 2 3 3 1 3 3 1 19

5.3% 10.5% 10.5% 15.8% 15.8% 5.3% 15.8% 15.8% 5.3% 100.0%

Total 1 6 4 12 5 4 6 4 1 43

2.3% 14.0% 9.3% 27.9% 11.6% 9.3% 14.0% 9.3% 2.3% 100.0%



0 10 8 18 7 7 8 1 0 59

0.0% 16.9% 13.6% 30.5% 11.9% 11.9% 13.6% 1.7% 0.0% 100.0%

2.00 Control group

2 15 10 9 7 6 4 4 2 59

3.4% 25.4% 16.9% 15.3% 11.9% 10.2% 6.8% 6.8% 3.4% 100.0%

Total 2 25 18 27 14 13 12 5 2 118

1.7% 21.2% 15.3% 22.9% 11.9% 11.0% 10.2% 4.2% 1.7% 100.0%

There is no significant difference between the two groups by gender. P=.337, p=.496

21

Secondary Schools: Comparison of Control & Treatment Groups

Table 6. Secondary School Students ELP

Group Treatment or Control group * GESEgrade

GESEgrade

Total 0 1 10 2 3 4 5 6 7 8 9



253 326 1 129 97 46 13 14 7 6 0 892

28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%

2.00 Control group

219 278 4 134 99 46 19 13 5 2 4 823

26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%

Total 472 604 5 263 196 92 32 27 12 8 4 1715

27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%

There is no significant difference between the two groups. P=.228

22

Table 7. Secondary School Students ELP disaggregated by Gender

Group Treatment or Control group * GESEgrade * Gender

Gender

GESEgrade

Total 0 1 10 2 3 4 5 6 7 8 9



163 169 1 72 61 31 10 9 6 4 526

31.0% 32.1% .2% 13.7% 11.6% 5.9% 1.9% 1.7% 1.1% .8% 100.0%

2.00 Control group

144 148 0 69 50 24 7 3 0 0 445

32.4% 33.3% 0.0% 15.5% 11.2% 5.4% 1.6% .7% 0.0% 0.0% 100.0%

Total 307 317 1 141 111 55 17 12 6 4 971

31.6% 32.6% .1% 14.5% 11.4% 5.7% 1.8% 1.2% .6% .4% 100.0%



90 157 0 57 36 15 3 5 1 2 0 366

24.6% 42.9% 0.0% 15.6% 9.8% 4.1% .8% 1.4% .3% .5% 0.0% 100.0%

2.00 Control group

75 130 4 65 49 22 12 10 5 2 4 378

19.8% 34.4% 1.1% 17.2% 13.0% 5.8% 3.2% 2.6% 1.3% .5% 1.1% 100.0%

Total 165 287 4 122 85 37 15 15 6 4 4 744

22.2% 38.6% .5% 16.4% 11.4% 5.0% 2.0% 2.0% .8% .5% .5% 100.0%



253 326 1 129 97 46 13 14 7 6 0 892

28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%

2.00 Control group

219 278 4 134 99 46 19 13 5 2 4 823

26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%

Total 472 604 5 263 196 92 32 27 12 8 4 1715

27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%

There is significant difference within the male group. P=.005

23

Table 8. Secondary Students ELP disaggregated by Division

Group Treatment or Control group * GESEgrade * Division Crosstabulation

Division

GESEgrade

Total 0 1 10 2 3 4 5 6 7 8 9

Barishal Group Treatment or Control group


11 48 21 7 1 88

12.5% 54.5% 23.9% 8.0% 1.1% 100.0%

2.00 Control group

30 56 17 8 0 111

27.0% 50.5% 15.3% 7.2% 0.0% 100.0%

Total 41 104 38 15 1 199

20.6% 52.3% 19.1% 7.5% .5% 100.0%

Chittagong Group Treatment or Control group


34 57 28 31 3 0 1 154

22.1% 37.0% 18.2% 20.1% 1.9% 0.0% .6% 100.0%

2.00 Control group

3 36 40 45 18 4 3 149

2.0% 24.2% 26.8% 30.2% 12.1% 2.7% 2.0% 100.0%

Total 37 93 68 76 21 4 4 303

12.2% 30.7% 22.4% 25.1% 6.9% 1.3% 1.3% 100.0%

Dhaka Group Treatment or Control group


85 75 42 25 14 5 6 4 4 260

32.7% 28.8% 16.2% 9.6% 5.4% 1.9% 2.3% 1.5% 1.5% 100.0%

2.00 Control group

69 54 20 18 14 1 0 0 0 176

39.2% 30.7% 11.4% 10.2% 8.0% .6% 0.0% 0.0% 0.0% 100.0%

Total 154 129 62 43 28 6 6 4 4 436

35.3% 29.6% 14.2% 9.9% 6.4% 1.4% 1.4% .9% .9% 100.0%

Khulna Group Treatment or Control group


49 37 6 16 11 119

41.2% 31.1% 5.0% 13.4% 9.2% 100.0%

2.00 Control group

73 37 3 0 0 113

64.6% 32.7% 2.7% 0.0% 0.0% 100.0%

Total 122 74 9 16 11 232

52.6% 31.9% 3.9% 6.9% 4.7% 100.0%

24

Rajshahi Group Treatment or Control group


13 29 1 3 8 17 8 6 3 2 0 90

14.4% 32.2% 1.1% 3.3% 8.9% 18.9% 8.9% 6.7% 3.3% 2.2% 0.0% 100.0%

2.00 Control group

9 25 4 14 6 4 8 9 5 2 4 90

10.0% 27.8% 4.4% 15.6% 6.7% 4.4% 8.9% 10.0% 5.6% 2.2% 4.4% 100.0%

Total 22 54 5 17 14 21 16 15 8 4 4 180

12.2% 30.0% 2.8% 9.4% 7.8% 11.7% 8.9% 8.3% 4.4% 2.2% 2.2% 100.0%

Rangpur Group Treatment or Control group


24 63 23 10 0 0 1 121

19.8% 52.1% 19.0% 8.3% 0.0% 0.0% .8% 100.0%

2.00 Control group

28 46 26 19 4 3 0 126

22.2% 36.5% 20.6% 15.1% 3.2% 2.4% 0.0% 100.0%

Total 52 109 49 29 4 3 1 247

21.1% 44.1% 19.8% 11.7% 1.6% 1.2% .4% 100.0%

Sylhet Group Treatment or Control group


37 17 6 0 0 0 0 60

61.7% 28.3% 10.0% 0.0% 0.0% 0.0% 0.0% 100.0%

2.00 Control group

7 24 14 3 6 3 1 58

12.1% 41.4% 24.1% 5.2% 10.3% 5.2% 1.7% 100.0%

Total 44 41 20 3 6 3 1 118

37.3% 34.7% 16.9% 2.5% 5.1% 2.5% .8% 100.0%



253 326 1 129 97 46 13 14 7 6 0 892

28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%

2.00 Control group

219 278 4 134 99 46 19 13 5 2 4 823

26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%

Total 472 604 5 263 196 92 32 27 12 8 4 1715

27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%

There are significant differences between the two groups in most of divisions, except Barishal, Dhaka

25

Table 9. Secondary Teachers’ ELP

Group Treatment or Control group * GESEGrade012 Crosstabulation

GESEGrade012

Total 1 10 2 3 4 5 6 7 8 9



1 2 3 4 20 10 6 7 3 2 1 59

1.7% 3.4% 5.1% 6.8% 33.9% 16.9% 10.2% 11.9% 5.1% 3.4% 1.7% 100.0%

2.00 Control group

0 2 1 5 8 10 11 10 5 1 1 54

0.0% 3.7% 1.9% 9.3% 14.8% 18.5% 20.4% 18.5% 9.3% 1.9% 1.9% 100.0%

Total 1 4 4 9 28 20 17 17 8 3 2 113

.9% 3.5% 3.5% 8.0% 24.8% 17.7% 15.0% 15.0% 7.1% 2.7% 1.8% 100.0%

There is no significant difference between the two groups.

26

Table 10. Secondary Teachers’ ELP disaggregated by Gender

Group Treatment or Control group * GESEGrade012 * Gender Crosstabulation

Gender

GESEGrade012

Total 1 10 2 3 4 5 6 7 8 9



2 1 2 6 0 2 1 2 16

12.5% 6.3% 12.5% 37.5% 0.0% 12.5% 6.3% 12.5% 100.0%

2.00 Control group

1 0 3 3 2 2 0 0 11

9.1% 0.0% 27.3% 27.3% 18.2% 18.2% 0.0% 0.0% 100.0%

Total 3 1 5 9 2 4 1 2 27

11.1% 3.7% 18.5% 33.3% 7.4% 14.8% 3.7% 7.4% 100.0%



1 0 2 2 14 10 4 6 1 2 1 43

2.3% 0.0% 4.7% 4.7% 32.6% 23.3% 9.3% 14.0% 2.3% 4.7% 2.3% 100.0%

2.00 Control group

0 1 1 2 5 8 9 10 5 1 1 43

0.0% 2.3% 2.3% 4.7% 11.6% 18.6% 20.9% 23.3% 11.6% 2.3% 2.3% 100.0%

Total 1 1 3 4 19 18 13 16 6 3 2 86

1.2% 1.2% 3.5% 4.7% 22.1% 20.9% 15.1% 18.6% 7.0% 3.5% 2.3% 100.0%



1 2 3 4 20 10 6 7 3 2 1 59

1.7% 3.4% 5.1% 6.8% 33.9% 16.9% 10.2% 11.9% 5.1% 3.4% 1.7% 100.0%

2.00 Control group

0 2 1 5 8 10 11 10 5 1 1 54

0.0% 3.7% 1.9% 9.3% 14.8% 18.5% 20.4% 18.5% 9.3% 1.9% 1.9% 100.0%

Total 1 4 4 9 28 20 17 17 8 3 2 113

.9% 3.5% 3.5% 8.0% 24.8% 17.7% 15.0% 15.0% 7.1% 2.7% 1.8% 100.0%

There is no significant difference between the two groups by gender. P=.446, p=.238

27

Appendix 2: QE study data analysis tables for Classroom Practice Study

Primary Schools: Comparison of Control & Treatment Groups

Table 1. Primary Classrooms: Types of Activity Treatment or Control group Teacher talking Student talking Student reading Student

writing Student listen audio

Other activity

Treatment group % 54% 24% 5% 5% 1% 12%

N 53 53 53 53 53 53

Control group % 53% 23% 5% 5% 1% 13%

N 52 52 52 52 52 52

There is no statistically significant difference between the two groups in terms of type of activity in a classroom.

Table 2: Primary Classrooms: % of Talk in Bangla and English, by Teachers and Students


Teacher Talk

Bangla

Teacher Talk

English

Student Talk

Bangla

Student Talk

English


% 26.98% 73.02% 10.28% 89.72%

2.00 Control group % 33.89% 66.11% 15.50% 84.50%

Total % 30.49% 69.51% 12.95% 87.05%

There is no statistically significant difference between the two groups.

28

Table 3: Primary Classrooms: Types of Teacher Talk Treatment or Control group Presenting Organising Giving

feedback Asking

question

Treatment group % 42% 25% 17% 17%

N 53 53 53 53

Control group % 43% 21% 18% 18%

N 52 52 52 52


Table 4: Primary Classrooms: Language of Teacher Talk by Type Presenting Organising Giving feedback Asking question

Treatment or Control group English Bangla English Bangla English Bangla English Bangla

Treatment group % 79% 21% 65% 35% 78% 22% 68% 32%

N 53 53 48 48 47 47 46 46

Control group % 73% 27% 52% 48% 70% 30% 67% 33%

N 51 51 45 45 41 41 46 46


29

Table 5: Primary Classrooms: Types of Student Talk Single In pair Group Choiring


Treatment group % 88% 12% 88% 13% 96% 4% 93% 7%

N 43 43 8 8 9 9 48 48

Control group % 81% 19% 67% 33% 80% 20% 88% 12%

N 42 42 9 9 9 9 44 44


Table 6: Primary Classrooms: Language of Student Talk by Type Single In pair Group Choiring


Treatment group % 88% 12% 88% 13% 96% 4% 93% 7%

N 43 43 8 8 9 9 48 48

Control group % 81% 19% 67% 33% 80% 20% 88% 12%

N 42 42 9 9 9 9 44 44


30

Secondary Schools: Comparison of Control & Treatment Groups

Table 7. Secondary Classrooms: Types of Activity Treatment or Control group Total of

Teacher talking time

Total of Student talking time

Total of Student reading time

Total of Student writing time

Total of Student listen audio time

Total of Other activity time

Treatment group % 62% 18% 4% 4% 2% 11%

N 51 51 51 51 51 51

Control group % 62% 18% 3% 3% 4% 10%

N 36 36 36 36 36 36

There is no statistically significant difference between the two groups in terms of type of activity in a classroom.

Table 8: Secondary Classrooms: % of Talk in Bangla and English, by Teachers and Students


Teacher Talk

Bangla

Teacher Talk

English

Student Talk

Bangla

Student Talk

English


% 32.28% 67.72% 21.60% 78.40%

2.00 Control group % 43.67% 56.33% 23.13% 76.87%

Total % 37.88% 62.12% 22.36% 77.64%


31

Table 9: Secondary Classrooms: Types of Teacher Talk Treatment or Control group Presenting Organising Giving

feedback Asking

question

Treatment group % 52% 16% 13% 19%

N 51 51 51 51

Control group % 54% 14% 11% 21%

N 36 36 36 36


Table 10: Secondary Classrooms: Language of Teacher Talk by Type Presenting Organising Giving feedback Asking question

Treatment or Control group

English Bangla English Bangla English Bangla English Bangla

Treatment group % 70% 30% 56% 44% 74% 26% 70% 30%

N 51 51 38 38 44 44 49 49

Control group % 60% 40% 49% 51% 59% 41% 51% 49%

N 35 35 29 29 28 28 33 33

There is no statistically significant difference between the two groups for the language used when Presenting and Organising. There are statistically significant differences between the two groups, for the language used when Giving Feedback and Asking Questions.

32

Table 11: Secondary Classrooms: Types of Student Talk

There is no statistically significant difference between the two groups. Note: the numbers of cases of pair or group work were very small.

Table 12: Secondary Classrooms: Language of Student Talk by Type Single In pair Group Choiring

Treatment or Control group

English Bangla English Bangla English Bangla English Bangla

Treatment group % 83% 17% 86% 14% 40% 60% 87% 13%

N 46 46 7 7 16 16 24 24

Control group % 76% 24% 88% 13% 95% 5% 72% 28%

N 30 30 4 4 5 5 24 24


Note: the numbers of cases of pair or group work were very small.

Treatment or Control group Single In pair Group Choiring

Treatment group Mean 64% 5% 13% 18%

N 48 7 16 24

Control group Mean 56% 4% 6% 33%

N 35 4 5 24

33

Appendix 3: Qualitative Study Field Protocol & Guidance Notes

Guidance Notes on semi-structured interviews Extract

‘A semi-structured interview is a verbal interchange where one person, the interviewer, attempts to elicit information from another person by asking questions. Although the interviewer prepares a list of predetermined questions, semi-structured interviews unfold in a conversational manner, offering participants the chance to explore issues they feel are important …

…Semi-structured interviews (sometimes referred to as informal, conversational, or ‘soft’ interviews) are about talking with people but in ways that are self-conscious, orderly, and partially structured. Krueger and Casey (2000) explain that it is …

‘…about listening. It is about being open to hear what people have to say. It is about being nonjudgemental. It is about creating a comfortable environment for people to share. It is about being careful and systematic with the things people tell you.’

…Dunn (2005:80) explains:

“Structured interviews follow a predetermined and standardized list of questions. The questions are always asked in almost the same way and the same order.

At the other end of the continuum are unstructured forms of interviewing such as oral histories… The conversation in these interviews is actually directed by the informant rather than by the set questions.

In the middle of this continuum are semi-structured interviews. This form of interviewing has some degree of predetermined order but still ensures flexibility in the way issues are addressed by the informant”.

When conducting semi-structured interviews or focus groups it is possible to take notes or to audio/video record the discussion. I usually audio(tape) the proceedings. This allows me to focus fully on the interaction instead of feeling pressure to get the participants worlds recorded in my notebook (see Valentine, 2005).

Directly after the interviews I document the general tone of the conversation, the key themes that emerged and anything that particularly impressed me or surprised me in the conversation. Taking these notes, in a sense, is a form of data analysis… It is advantageous to transcribe interviews and focus groups as soon as possible after conducting them… Hearing the taped conversation whilst it is still fresh in your mind makes transcription much easier …’

34

Field Protocol General School Information 1. name, location and designation of the school

2. size of the school

• number of students on roll • student numbers by class, disaggregated by gender • teacher numbers (disaggregated by gender) • for each teacher being interviewed: Which classes do they teach? Which

subjects do they teach? (Primary: do all teachers teach all subjects). • does the school operate shifts? (How many, what times)? • number of classrooms?

3. other general information about the school.

4. How many Assistant Teachers, other than the two teachers included in the study, can and do teach English?

5. Professional development context

• existing or previous involvement of the school and its teachers in professional development offered by the Upazila (or external agencies in secondary) in last 3 years.

• are there any professional development / teaching quality improvement activities happening in the school (other than EIA)?

• does the head teacher have a role in professional development / teaching quality improvement activities in the school? Elaborate...

6. Does the school have links to local neighbouring schools (including primary-

secondary links)

Other additional information Photograph the school, particularly any additional resources, such as staff rooms, libraries, computer rooms, or any other unusual / interesting features.

35

SSI with Teachers Before the lesson observation and interview, where possible: • Record the teachers’ name and gender.

• Record the class (year) being observed (e.g. Class 7)

• If possible, photograph the classroom, teacher, and any resources (EIA or otherwise) to support English language teaching

• You may find it helpful to sketch the class (position of benches / tables / teachers desk / board / resources / students seating by gender)

During lesson observation record any observations you feel appropriate to support the discussion to follow. In particular: • Teacher behaviours related to communicative practices

• Use of activities, practices or materials that may have been promoted by Teacher Development Programmes (e.g. EIA, SBT, TQI etc).

• Novel or unexpected teaching and learning practices

• Student responses; extent and equity of participation

Time-stamp your notes (e.g. actual time, or lesson time) regularly. Discretely photograph or audio or video record any interesting practices during the lesson, if appropriate and possible without causing distraction / disruption.

SSI with EIA teachers, on New Classroom Activities 1. Open Ended

1. How do you feel the lesson went? (Was there anything in particular you were pleased with, or that you might do differently next time?)

2. How would you describe the students’ reaction to the lesson?

3. If I’d come here to see your lesson a year or two ago and come back now, how do you think those lessons would compare? (If there are important changes, what or who most helped you make them?)

4. If I come back again in a year or two’s time, what differences would you hope I’d see

a. in your lesson?

b. in you or your students?

What might be the most important thing in helping you make that change? 2. Directed to Themes (Note: Areas to discuss, not questions to ask!)

1. SBTD activities in the classroom

• What effect would you say any professional development or training programmes have had on what you or your students do in the classroom?

36

– Can tell me about any specific examples from the lesson we’ve just seen, or from your teaching in the last week or so?

– If so, what was there anything that helped you make that change / learn how to do this?

– What do you think you learned from making that change?

• Is there anything that makes it hard to introduce ideas from training programmes into your classroom practice, and what kinds of things might help overcome these difficulties?

2. SBTD activities outside the classroom

• Apart from attending training courses or upazila meetings, is there anything you do at home or school, that helps you get new ideas for teaching or learning? (observations, books, materials, online / offline resources etc. etc.).

• Can you give any specific examples of something you learned like this… can you tell me how you learned it…

• If another teacher in your school wanted to improve their English teaching, apart from going on courses, would you have any advice for them, based on your experience?

3. SBTD activities with peers in school

• Are there any ways in which other teachers from this school have helped you think or act differently in the classroom?

• How do the kind of routines and relationships that exist in the school support or restrict this kind of learning between teachers

4. The Role of the Head Teacher or EO’s in promoting / monitoring SBTD in school

Can you think of anything that the head teacher or your local education officers have done in your school, that help you and other teachers improve teaching and learning in your lessons?

37

SSI with Head Teacher

Head Teacher Interview A. Open Ended

1. Check and update the general information (see p4).

2. What do you think is the most important role of the head teacher – and what do you spend most of your time doing?

3. What are you trying to achieve for the school?

B. Directed to Themes

4. Do you have a particular approach (strategy) that you hope to bring about the

improvement of your teaching staff? Probe through asking:

• How particular staff chosen to participate in or lead development activities? • Do individual teachers make their own choices, are is participation required? • What is the head teacher’s role and how do they fulfil it?

5. Explore HT views related to question 5 (p4) – the professional development context

of the school.

• What kinds of things are happening in the school, to improve classroom practices… how does the HT promote or monitor such changes?

• Is there anything that happens in school, to help teachers learn about new approaches, or learn from each other? What’s the head teacher’s role in this?

• Are there any particular teachers that have done something to improve teaching and learning in the school… what did they do, and why did they do it? How did students / other teachers / the HT / the community respond?

6. What are the biggest challenges to improving teaching and learning in the school?

Is there anything helping or hindering you or your teachers, in overcoming these?

7. Can you give any specific examples of ways in which your Education Officer has promoted or monitored changes in teaching and learning in your school X, in the last 3 months?

38

SSI with Education Officer

A. Open Ended

1. What do you think is the most important role of the Education Officer – and what do you spend most of your time doing?

2. What are you trying to achieve for the schools in your Upazila?

3. Do you have any particular hopes or challenges in relation to school X (that we are studying).

B. Directed to Themes 4. What’s your strategy for helping teachers in school X improve teaching and learning

in their classrooms?

5. How do you encourage teachers to put what they learn on training courses or in local meetings, into practice in the classroom? How do you know the extent to which this is happening in reality?

6. How would you describe the head teachers role, in relation to improving quality of teaching and learning in their school, or helping teachers put their training into practice?

7. How do you see your role, in relation to the head teachers role? Practically, what mechanisms do you have to support, guide or monitor head teachers, and how do those work.

8. Can you give any specific examples of ways in which you have promoted or monitored changes in teaching and learning in school X, in the last 3 months?

progress report: eia phase iv studies - eiabd.com · progress report: eia phase iv studies ... and...

Documents