Final Report

Evaluation of School-Based INSET Pilot Programme

Frank Hardman and Hillary Dachi

December 2012

2

ACKNOWLEDGEMENTS This study was made possible by the support and contributions from various individuals, groups and institutions. Specifically we would like to thank the MOEVT, PMO-RALG and UNICEF for coordinating the study and the District Councils of Bagamoyo, Hai, Kwimba, Magu, Makete, Mtwara(R), Siha and Temeke for supporting and hosting the teams during the data collection phase. Special thanks go to Maniza Ntekim and Audax Tibuhinda from UNICEF and Sara Kironde from MoEVT and PMO-RALG for their assistance and support throughout the study. The MOEVT, PMO-RALG and UNICEF also made the following officials and Teacher College tutors available for the data collection phase: Friedrick Shuma Nicholaus Moshi Madina Kemilembe Anna. N. Mbassa Martha. G. Quaresi Hamza Salehe Sheshe Godson Lema Beatrice .N. Mkina Anders Mutembei Andrew L. Binde Therese S.K. Mbuligwe Rosemary Massam Penina Sangiwa Audax Tibuhinda Placid. M. Balige Newako Kasunga T. Masota Anastazia Msami The study team is very grateful for their support in terms of the commitment and high standards of professionalism shown. During the field work District Education Officers, INSET coordinators, Ward Education Coordinators, Head teachers, Teachers, Pupils and School Committee members, College Principals and Tutors played an important role in the study and we would also like to extend our thanks to them. The University of York study team comprised the following: Frank Hardman Isabel Bjork Emily Doie Louise Elliot Sarah Hogben Solvor Lauritzen

3

CONTENTS List of tables List of figures Acronyms and abbreviations 1. EXECUTIVE SUMMARY 7 1.1 Introduction 7 1.2 Scope of the Evaluation 8 1.3 Design of the study 8 1.4 Findings 9 1.5 Moving forward 12 1.6 Conclusions 14 2. BACKGROUND TO THE STUDY 16 3. METHODOLOGY 21 3.1 Introduction 21 3.2 Research questions 21 3.3 Research design 21 3.4 Sample 22 3.5 Research instruments 22 3.6 Roles and responsibilities 26 3.7 Timeline 27 4. FINDINGS 28 4.1 Introduction 28 4.2 Analysis of classroom observation data 28 4.3 Analysis of interview 34

5. CONCLUSIONS 44 6. EMERGING LESSONS 46 APPENDICES 48

4

List of tables

Table 1: Class size 22 Table 2: Timeline for study 27 Table 3: Breakdown of observed lessons by subject and year group 28 Table 4: Percentage of time spent on teaching and learning activities in 30

intervention schools Table 5: Percentage of time spent on teaching and learning activities in 31

control schools

5

List of Figures

Figure 1: Intervention Group Timeline Analysis 32 Figure 2: Control Group Timeline Analysis 33

6

Acronyms and abbreviations

ADEM Agency for the Development of Education Management D by D Decentralization by Devolution DEDs District Executive Directors DEOs District Education Officers DUCE Dar es Salaam University College of Education IAE Institute of Adult Education INSET In-service Education and Training IRF Initiation-Response-Feedback LGAs Local Government Authorities MOEVT Ministry of Education and Vocational Training MOF Ministry of Finance OECD Organization for Economic and Cooperative Development OUT Open University of Tanzania PEDP Primary Education Development Plan PMO-RALG Prime Ministers’ Office–Regional Administration and Local Government PRESET Pre-service training SEDP Secondary Education Development Plan TCs Teacher Colleges TDMS Teacher Development and Management Strategy TESSA Teacher Education in Sub-Saharan Africa TIE Tanzania Institute of Education TRCs Teacher Resource Centres UDSM University of Dar es Salaam UNICEF United Nations Children’s Emergency Fund

7

EXECUTIVE SUMMARY 1.1 INTRODUCTION This report provides the findings of an evaluation of the impact of the pilot In-service Education and Training (INSET) strategy for Tanzanian primary school teachers, which was approved in 20091 and a pilot launched in early 20112. In its commitment to raising the quality of basic education and achieving the Dakar Framework on Education for All and the Millennium Development Goals, the Government of Tanzania embarked on major education reforms. The Government developed and implemented the Primary Education Development Plan (PEDP: 2002-2006) and the Secondary Education Development Plan (2004-2009). During the implementation of these plans it became evident that teacher education needed to be strengthened to respond positively to the demands of the PEDP and SEDP plans. Thus, the Teacher Development and Management Strategy (TDMS 2008 - 2013) was formulated and launched in 2008.

The goal of the TDMS was ‘to have and sustain adequate numbers of competent teachers and tutors to effectively support the pre-primary, primary, secondary, adult and non-formal education, as well as Teachers’ Colleges’. TDMS was designed to improve the quality of Pre-Service Teacher Education (PRESET) and In-Service Teacher Education and Training (INSET) as an important step in the Government’s efforts to provide a quality basic education for all. Following a baseline study conducted in 2008 that found primary teaching was largely made up of teacher-led rote, recitation and exposition and INSET often uncoordinated, ad-hoc and of varying quality, the government decided to concentrate its efforts on developing a quality, sustainable, cost-effective INSET system for primary school teachers.

In 2009 the Ministry of Education and Vocational Training (MOEVT) approved the INSET strategy for primary schools teachers which included a monitoring and evaluation framework for implementation of the strategy. The strategy was approved following a highly consultative process which involved ministries, departments and agencies, development partners and civil society organisations. Supported by UNICEF, in 2011 the MOEVT and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG) began piloting the new school-based INSET model in seven Councils (Bagamoyo, Hai, Magu, Makete, Mtwara(R), Siha and Temeke) with a view to implementing the INSET strategy nationally. Modules for teachers were developed in three key areas: mathematics, English and pedagogy. Based on a nationally agreed teacher competency framework and drawing on national and international research into effective teaching and learning theory the modules were designed to promote teaching as an interactive, problem-solving activity,

1 MOEVT. 2009. In-service Education and Training Strategy for Primary School Teachers 2009 - 2013.

Ministry of Education and Vocational Training, Dar Es Salaam. 2 The Citizen, 25/02/2011.

8

requiring a thoughtful and reflective approach to classroom practice. The modules used blended learning systems which allowed for face-to-face teaching, supported self study and cluster support so as to create a community of learners in the school and to give teachers ownership of the INSET process. The modules also promoted coaching, mentoring, classroom observation and feedback from peers, head teachers, college tutors and ward coordinators so that teachers study their own classes and that of other teachers to discuss outcomes for improvement. The English and mathematics modules specifically focused on the teaching of literacy and numeracy so as to develop a broader and deeper understanding of reading and mathematics from lower primary through to upper primary. To date 2,052 primary school teachers from 141 schools across the 7 Districts have been participating in the pilot programme. 1.2 SCOPE OF THE EVALUATION The purpose of the evaluation as set out in the Terms of Reference (Appendix 1) was to:

Assess the effectiveness and efficiency of the pilot programme, particularly with regards to improving teaching methodologies; enhancing knowledge of the subject matter by teachers and; making a difference to learning experiences, and where possible learning outcomes, of children;

Identify gaps within the current INSET model with a focus on: a) the INSET delivery system itself, including the support mechanisms for teachers; and b) the content of INSET programme materials;

Provide lessons learnt and clear concrete conclusions on how to improve or tweak the existing model with a view to national roll-out, and;

Assess whether communities and parents seen any changes in how their children are learning?

1.3 DESIGN OF STUDY Using a mixed-method design to collect both quantitative and qualitative data, the study set out to assess the effectiveness and efficiency of the pilot programme particularly with regards to improving teaching methodologies and enhancing subject knowledge of the teachers and making a difference to the learning experiences, and where possible, learning outcomes of children. It also set out to identify gaps within the current INSET model with a focus on the INSET delivery systems, including the support mechanisms for teachers and the content of INSET programme materials with a view to improving the existing model before it is scaled up nationally. In order to measure the effectiveness of the programme in bringing about changes in pedagogic practice, the study used 3 systematic classroom observation schedules to measure differences between an intervention group of schools who had participated in the INSET and a control group who had not received the training. The data were also compared to a 2008 baseline study of teaching and learning practices and INSET provision in Tanzanian primary schools. The first observation instrument recorded the frequency of use of effective teaching behaviours drawn from the international literature, the second used a timeline analysis framework to record the time spent on a range of teaching and

9

learning behaviours, and the third recorded the types of discourse moves used by used by teacher and pupils in sub-sample of digitally recorded English lessons. In order to measure the efficiency of the programme systems for delivering school-based INSET, semi-structured interviews were conducted with key stakeholders at the national, regional, district and school level. 1.4 FINDINGS Based on an analysis of the observation data, the following findings emerged with regard to the impact of the INSET programme on pedagogic practices:

1. Compared to the control and baseline study, it was found that teachers who had participated in MOEVT INSET training showed statistically significant differences in the way they introduced and development lessons, used teacher questioning and feedback, and managed classes.

2. INSET trained teachers were significantly more likely to involve the pupils in group work and pupil demonstration, and to spend less time on class management issues. There were also fewer interruptions to their lessons and pupils were more likely to be on-task. Teaching and learning activities were more evenly spread across lessons suggesting higher levels of teacher-pupil interaction throughout the lesson.

3. Significant differences were also found in INSET trained teachers’ use of open questions and pupil demonstration. They were also more likely to call on an individual pupil to answer a question rather than cue a choral answer and to probe, comment and build a pupil answer into a subsequent question and to encourage pupil questions.

4. When asked about their current level of motivation for teaching, 33 per cent of teachers who had participated in the INSET stated they were ‘very motivated’, 55 percent stated they were ‘on the whole motivated’ and just 12 percent stated they were ‘not at all motivated’.

Based on the interviews, the following findings emerged with regard to the effectiveness and efficiency of the school-based INSET delivery systems: Teachers

1. Three quarters of the teachers interviewed stated they found the study modules very useful with a third specifically referring to the pedagogy module. However, 20 percent stated they found the English module more difficult than the mathematics and pedagogy modules that were written in Kiswahili.

2. Sixty percent of the teachers found the cluster group meetings and study groups to be very effective and 40 percent found the classroom observations to be very useful.

10

3. 70 percent of teachers reported the INSET had improved pupil participation in lessons and broadened their teaching repertoire through a greater use of paired and group work, and teaching aids drawn from the local environment.

4. Over a half of the teachers reported the INSET had improved lesson planning and helped teachers to identify the learning needs of the pupils they teach.

5. Sixty percent of teachers stated they had undertaken most of their study in the evenings after school with 30 percent meeting in study groups in the school. Nearly a third of the teachers said they studied during school hours during lunchtime and breaks, and a similar number stated they studied at weekends.

6. Seventy percent of teacher reported they had been observed. A third indicated they had been observed by a peer, 24 percent by a head teacher, 18 percent by a college tutor and 17 percent by a ward education coordinator.

7. In Mtwara, all 21 teachers interviewed stated they had been observed and 16 had been observed by a college tutor. In Bagamoyo 5 out of 16 teachers had been observed by a tutor and in Magu it was 2 out of 16. However, in Hai, Makete, Siha and Temeke none of the teachers had been observed by a college tutor.

8. Over a third of teachers raised concerns about the timing of feedback from college tutors on assignment.

9. The overwhelming majority of teachers interviewed thought more time should be made available for school-based INSET through the allocation of training days and study time during the working week.

10. Forty percent of teachers wanted more face-to-face training, cluster support and follow up in the classroom.

11. Over 90 percent of teachers stated they wanted the training to be accredited and rewarded through promotion.

12. Overall, the teachers were very positive about the training they had received and thought it should be scaled up at the national level to include all teachers.

Head teachers

1. There was a general agreement amongst the head teachers that the pilot INSET had impacted on teacher and pupil performance, creating a more learner-centred form of teaching and learning, a greater confidence in teachers and students alike, and greater subject matter competence from teachers and pupils.

2. Mentoring, cluster groups, study groups, observations and reading of the modules by the teachers, were deemed to be the most important features of the programme.

11

3. Eighty percent of the head teachers reported their teachers had been observed,

with peer review playing a major role. Some head teachers indicated, however, that there was a need for more external input into the observation of teachers, particularly from college tutors.

4. Ninety percent of head teachers reported they had a whole school development plan and a quarter stated it featured INSET as a priority; however most of those indicated they could not implement the plan due to a lack of an INSET budget.

5. Head teacher were unanimous about the need for providing incentives such as certificates and additional pay and promotion for those teachers who participate in the INSET.

School Committees

6. While most School Committees reported they knew about the INSET it was only

in a general sense, with most aware that that the programme had to do with teacher training and that it involved teachers working together in the school and at cluster meetings.

7. Despite a lack of detailed knowledge on how the programme was working, most of the School Committees (24 out of 28) agreed that the INSET had improved the quality of teaching in their schools through improved teacher-pupil relations and higher levels of commitment from the teachers.

Pupils

1. Pupils in the intervention schools reported teaching practices had changed, particularly in the use of group work, teaching and learning aids and pupil participation in lessons.

2. All pupil focus groups reported that teachers were friendlier, they had more time for the pupils and they allowed pupils to ask questions.

3. They also reported that lessons were more varied and that group work was a daily feature of their lessons, particularly in mathematics, English and science.

Teacher Colleges

1. Teacher College principals and tutors were generally positive about the design of

the programme and the modules, and thought it had improved teacher effectiveness and pupil learning. They also thought the programme was time efficient as teachers could study the modules in their own workplace supported by school study groups and cluster-based meetings and follow up in the classroom.

12

2. The main constraint identified by college principals and tutors was the lack of an INSET budget to cover travel costs to the schools, cluster meetings and district education offices, thereby limiting the number of visits they could make to mentor, monitor and assess the teachers and to liaise with the District Councils.

3. Principals and tutors felt there was a need for more training of teacher educators, including ward education officers, district INSET coordinators, district inspectors and cluster coordinators.

District Education Officials

1. There was general agreement amongst district level education officers (i.e. District

Education Officers, Ward Education Coordinators, District Inspectors, District INSET Coordinators and Ward Education Coordinators) that the INSET training had improved teacher competence, subject knowledge and confidence leading to more learner-centred approaches.

2. The majority of education officials working at the district level thought that that the modules, particularly the pedagogy module, had been effective in training the teachers in their own schools and that the mentoring, observation, school and cluster meetings had provided a good level of support.

3. District education officials viewed the school-based INSET training as being cost effective because the distance learning materials ensured teachers were not taken away from their schools and they could study in their own time during the school day, in the evenings and at weekends.

4. The lack of an INSET budget being made available at the district, ward and school level for school-based training was seen by district education officers as a major challenge for the sustainability of the INSET programme which they felt should be rolled out to all teachers.

5. District education officials generally agreed that the INSET programme needed accreditation and that teachers who took part in the INSET needed to be rewarded with higher pay and promotion.

6. District education officials felt that there was a need for further training, accreditation and career advancement of teacher educators working at the school level to help build their capacity for delivering, mentoring and monitoring school-based INSET

1.5 MOVING FORWARD

1. Based on the findings with regard to changes in pedagogic practice and general support amongst stakeholders for the effectiveness and efficiency of the school-based INSET, the programme should be scaled up and made available to all Tanzanian primary school teachers.

13

2. The national scale up should be phased and rolled out gradually over a specified

period of time. One district from each of the regions should be selected on the basis of mutually agreed criteria to become a model district for other districts. A few wards, clusters and schools should be selected and supported to become centres of excellence in terms of cluster coordination, expert teachers, peer learning and networking.

3. As in the pilot, the national scale up of school-based INSET will require a clear division of roles and responsibilities between national, regional and district offices, and between head teachers, schools, tutors and mentor colleges. It will also require the development of an effective communication strategy to ensure all stakeholders are fully briefed about its implementation.

4. Budgets specifically for school-based INSET should be devolved down to the college, district and school level.

5. The capacity and training needs of those charged with organising and providing

the training, mentoring and coaching for school-based INSET, such as district education officers and college tutors, ward education officers, district cluster leaders and inspectors, needs to be made a major priority in scaling up school-based INSET.

6. Incentives and accreditation for teachers who complete the INSET training need to be put in place.

7. Official staff training days should be made available to schools throughout the school year to support school-based INSET.

8. Mentor Teacher Colleges should be given the budget resources to ensure they can support the district offices, schools and clusters in providing school-based INSET and provide timely feedback on classroom observations and course work assignments.

9. The INSET modules should be made available to all Teacher Colleges for use PRESET programmes. In addition to their current print form, they should be made open and freely available in a variety of electronic formats such as compact discs and data sticks. After the national scaling up of the INSET they should be supported by an electronic platform such as websites (internet), television and radio broadcasts.

10. More rigorous monitoring systems should be put in place at the national, regional and district level to systematically collect and use data on learning achievement to measure the impact of the INSET training.

11. While initial costing of the INSET pilot suggests there are economies of scale compared to traditional college-based provision due to the wide reach of the

14

distance education programme and its ability to reach remote area, a more rigorous cost benefit analysis of PRESET and INSET should be carried out during the national roll out to inform future developments in teacher education. Such data can be used to bring about a better balance between the time and money spent on pre-service and subsequent in-service training.

12. Management and financial mechanisms need to be put across all levels of the education system to help improve accountability, planning and implementation, and ensure transparency in INSET budgets allocations.

1.6 CONCLUSIONS Overall, the findings from the school-based INSET pilot support the view that enhancing the capacity of the teaching profession is crucial if the quality of teaching and learning in primary schools in Tanzanian is to improve. In terms of the effectiveness and efficiency of the delivery systems, it was generally agree across all stakeholders at the national, regional, district and school level that the study modules, classroom observations, school-based study group and ward cluster meetings had been very effective in broadening the teaching and learning repertoire and in improving the quality of primary education. It was reported that it had impacted on teacher and pupil performance, creating a more learner-centred form of teaching and learning, a greater confidence in teachers and students alike, and greater subject matter competence from teachers and pupils. This perception was supported by the findings of the classroom observations showing school-based training can help teachers develop more of a dialogic pedagogy so as to promote cognitive engagement and understanding. While the findings showed statistically significant shifts in pedagogic practices and the majority of stakeholders interviewed believed it had improved the quality of teaching and learning, there is a need for a more systematic monitoring of learning outcomes using pre and post-testing during the next phase of the of the scale up in both intervention and control groups of schools. In adopting the school-based model of training, initial costing of the INSET programme suggest there are economies of scale due to the wide reach of the distance education programme and its ability to reach remote areas. In contrast to traditional college-based provision, the pilot INSET programme allowed for the integrated the use of distance education and face-to-face delivery in a flexible model supported by classroom observation and teachers coming together in study group and clusters to share and reflect on the experiences. Crucially, it allowed the teachers to study in their own schools, thereby ensuring a better blend of theory and practice and promote a sense of professionalism so as to raise educational achievement. Although the findings of the evaluation are overwhelmingly positive, it identified some areas in need of attention with regard to the support mechanisms, capacity building, budget disbursement and accreditation of the school-based training. There was unevenness in the range of observations teachers received from teacher educators, including college tutors, district education offices, ward coordinators, district inspectors, and INSET coordinators across districts and in the timing of feedback on module

15

assignment. The overwhelming majority of head teachers and teachers interviewed also thought official time should be made available for school-based INSET throughout the school year through the provision of training days. There was also general agreement amongst teachers, head teachers and district education officials that the INSET should be accredited and those teachers who complete the programme should be rewarded with pay and promotion. While MOEVT officials, college principals and tutors, district education officials and head teachers thought the programme was time efficient as teachers could study the modules in their own workplace, supported by school study groups and cluster-based meetings and follow up in the classroom, problems with disbursement of budgets for school-based INSET at the regional, district, college and school level was seen as a major challenge. The main constraint identified by those working at the district and school level was the lack of an INSET budget to cover travel costs to the schools, cluster meetings and district education offices, thereby limiting the number of visit that could be made to schools to mentor, monitor and assess the teachers and to planning and implementation of the programme. The capacity building of teacher educators charged with delivering school-based INSET, including college tutors, ward education officers, district INSET coordinators, district inspectors and cluster coordinators, was also seen as a priority. This could be achieved through further training, accreditation and career advancement of teacher educators working at the school level to help build their capacity for delivering, mentoring and monitoring school-based INSET

16

BACKGROUND TO THE STUDY This section reports on the background to the study in terms of its rationale and design and draws on recent evidence to support the development of school-based training from the international literature into teacher development and a baseline study of teaching and learning and INSET conducted in Tanzania in 20083. In common with other countries in sub-Saharan Africa, Tanzania has expanded its primary education provision in order to achieve universal primary education by 2015. Given the need to address the quality of a rapidly expanding teaching force, the MoEVT set about the development of a national INSET strategy as part of the Teacher Development and Management Strategy (TDMS, 2008 – 2013). To inform the design of the TDMS, a mapping of existing teacher education related policies, structures, plans and activities was conducted, together with the baseline study of classroom processes and INSET provision and review of teacher education4. The baseline study and review of the Teacher Colleges found INSET in the 7 districts of Bagamoyo, Hai, Magu, Makete, Mtwara, Siha, Temeke and Shinyanga to be ad hoc. Overall, it was found just over half the sample had undertaken INSET or upgrading in the last 5 years. Most of the recorded INSET was of a short duration, lasting no more than two weeks. However, across the districts there was a large degree of variation: only 5 percent and 9 percent of teachers in Temeke and Bagamoyo respectively reported that they had taken part in INSET or upgrading programmes compared to 86 per cent in Siha and 84 per cent in Shinyanga. From these studies, guidelines for the development of an INSET strategy linked to the TDMS were developed. The INSET programme was officially launched in February 2011 (The Citizen, 25/02/2011). As the 2009 INSET strategy recognises, the quality of an education system depends to a large extent on the quality of its teachers, as they are the key source of knowledge and skills. International research into classroom processes recognises that managing the quality of teacher-pupil interaction is one of the most important factors in improving the quality of teaching and learning, particularly in contexts where learning resources and teacher training are limited5. Such research argues it is possible to pinpoint universals in teaching and learning, such as teacher-pupil and pupil-pupil interaction in whole class, group based and one-to-one teaching and learning situations which must be attended to so as to improve the quality of education. Helping teachers to transform classroom talk into a purposeful and productive dialogue, through a pedagogy and curriculum which is relevant to the lives and linguistic profile of the communities from which the pupils come, is therefore seen as being fundamental to improving the quality of primary education in Tanzania.

3 UNICEF. 2008. The Quality of Teaching and Learning in Tanzanian Primary Schools: A Baseline Study.

UNICEF, Dar Es Salaam. 4 UNICEF.2009. A review of teacher education in Tanzania and the potential for closer links between PRESET

and INSET. UNICEF, Dar Es Salaam. 5 Hardman F. 2011, A Review of Save the Children’s Global Teacher Support and Development Interventions.

Save the Children Global Alliance, London.

17

Such developments in teacher education in Tanzania are also supported by the international literature into teacher development and support. In its most recent review of teacher education covering 65 countries from around the world, the Organisation for Economic and Cooperative Development (OECD) reported that in the high-performing education systems teachers have a central role to play in improving educational outcomes, and are also at the centre of the improvement efforts themselves6. Such systems are not driven by top-down reforms but by teachers embracing and leading on reform, taking responsibility as professionals, thereby developing a wider repertoire of pedagogic strategies for use in the classroom. The OECD study also found that the most effective professional development programmes upgrade pedagogic knowledge and skills over a sustained period of time rather than through disjointed one-off courses. They bring together initial teacher education, induction and continuing professional development so as to create a lifelong framework for teachers. In this way, high performing education systems provide opportunities for teachers to work together on issues of instructional planning, to learn from one another through mentoring or peer coaching and by conducting research on the outcomes of classroom practices to collectively guide curriculum, assessment and professional learning decisions. The high performing education systems also benefit from clear and concise profiles of what teachers are expected to know and be able to do at different stages of their careers so as to guide initial teacher education, induction and professional development and create a lifelong learning framework for teachers7. The establishment of such benchmarks to assess progress in professional development over time means that appraisal and feedback are used in a supportive way to recognise and reward good performance. Because teaching is a complex activity in which moment-by-moment decisions are shaped by teacher beliefs and theories about what is effective teaching, theory and practice must be carefully integrated. In effective professional development, theories of curriculum, effective teaching and assessment are developed alongside their application in the classroom. Such integration allows teachers to use their theoretical understandings as a basis for making ongoing, principled decisions about practice. Focusing only on skills will not develop the deep understanding needed if teachers are to change their beliefs and practices and meet the complex demands of everyday teaching. Conversely, merely teaching theoretical constructs to teachers without helping to translate them into classroom practice will also prove ineffective. Challenging and changing beliefs and classroom practices also requires the development of self-regulatory skills that enable teachers to monitor and reflect on the effectiveness of the changes they make to their classroom practice. Such change appears to be promoted by a cyclical process of professional learning in which teachers have their current assumptions challenged by the demonstration of effective practice, develop new

6 OECD. 2011. Building a High Quality Teaching profession: Lessons From Around the World. Paris: OECD

publishing. 7 Darling-Hammond, L., Chung Wei, R., Andree, A., Richardson, N. & Orphanos, S. 2009. Professional

Learning in the Learning Profession: A Report on Teacher Development in the United States and Abroad. Stanford University/National Staff Development Council.

18

knowledge and skills, make small changes to practice aided by classroom observation, and observe resulting improvements in student learning outcomes. It also requires teachers being brought together in professional learning communities and informed by expertise external to the group of participating teachers The stakes for improving the quality of education through teacher development are particularly high in low income countries like Tanzania as research suggests that teaching is the most important factor in student achievement8. In sub-Saharan Africa the teacher is said to account for 27% of student achievement which is much higher than in high income countries. It also reflects the fact that in the absence of textbooks and other learning resources the teacher is the primary source for learning academic content and therefore key to improving the quality of education in resource poor environments. This is because educational quality is largely obtained through pedagogical processes in the classroom and that what students achieve is heavily influenced by the knowledge, skills, dispositions and commitment of the teachers in whose care students are entrusted. Intervening at the school and classroom level through school-based INSET in Tanzania is therefore crucial in raising the quality of teaching and learning in schools, as ultimately educational quality is obtained through pedagogical processes in the classroom: through the knowledge, skills, dispositions and commitments of the teachers in whose care pupils are entrusted. Developing the quality of classroom interaction between teachers and pupils through in-service training was therefore been identified as a key priority in the 2009 INSET strategy. Studies of pedagogy in primary schools from around the world show teachers often rely on a single method made up of teacher-fronted ‘chalk and talk’ promoting the transmission of knowledge and rote learning9. Such interaction often takes the form of lengthy recitations comprising of teacher explanation and questions, and brief answers often chorused by the whole class or by individual pupils. Similarly, the 2008 baseline study of classroom processes in Tanzanian primary schools found10:

lessons were dominated by lecturing punctuated by cueing of choral responses, question and answer routine, pupils copying from the chalkboard, written exercises and teachers marking pupil work

the vast majority of questions were ‘closed’ (i.e. calling for a single response or offering facts) as opposed to ‘open’ (i.e. calling for more than one answer): open-ended questions accounted for only 2% of the total

pupil questions were rare making up 1% of the questioning exchanges

boys were nearly twice more likely to be asked a question by the teacher than girls

over 60 percent of pupil responses were given by choral response

8 Dembele, M. & Lefoka, P. 2007. Pedagogical Renewal for Quality Universal Primary Education: Overview

of Trends in Sub-Saharan Africa. International Review of Education, 53: 531 – 553. 9 Hardman, F. & Abd-Kadir, J., 2010. Classroom discourse: towards a dialogic pedagogy. In: Wyse, D.,

Andrews, R., Hoffman, J (Eds.), The International Handbook of English, Language and Literacy. Routledge, Taylor and Francis, London, pp. 254 – 264 10

UNICEF 2008. The Quality of Teaching and Learning in Tanzanian Primary Schools: A Baseline Study. UNICEF, Dar es Salaam.

19

feedback to a pupil response was often absent, particularly in choral responses, non-committal or followed-up with a simple affirmation

paired/group work was observed in only 3% of lessons

96% of lessons used a traditional classroom layout with desks set out in rows facing the chalkboard

teaching and learning resources were very scarce with an average pupil/textbook ratio of pupil/textbook ratios of 11:1 at Grade 3 and 7:1 in Grade 6. This tended to reinforce practices whereby the teacher spent an excessive time writing up notes on the blackboard

Overall, the classroom pedagogy in Tanzanian primary schools was often made up of rote learning and memorisation rather than the acquisition of skills and the development of understanding in terms of higher order cognitive abilities. International research suggests changing such a narrow repertoire of pedagogic practices by managing the quality of classroom interaction can be a cost effective way of improving classroom pedagogy11. As the INSET strategy acknowledges, helping teacher educators and teachers transform classroom talk from the familiar rote, recitation and exposition to include a wider repertoire of dialogue and discussion in whole class, group-based and one-to-one interactions will require providing in-service education and training in alternative classroom interaction and discourse strategies that are more pupil-focused and dialogic in nature. By focusing on the classroom, the INSET strategy recognises that school-based training can help teachers develop more of a dialogic pedagogy to broaden the repertoire of whole class teaching currently found in Tanzanian primary classrooms. In this way dialogue and discussion can be included alongside the more traditional drilling, closed questioning and telling, thereby raising cognitive engagement and understanding. Such an approach builds on the traditional model of whole class teaching which is found in many Tanzanian primary classrooms but avoids the simplistic polarization of pedagogy into ‘teacher-centred’ versus ‘child-centred’ that has characterised much of the educational discourse in the international donor community12. It will also help to ensure there is a better balance and blending of local cultural practices with internationally informed teacher education reforms13. As well as promoting more active forms of learning, the development of the strategy also supports recent developments in the introduction of a competency-based curriculum in Tanzania supported by the development of school-based modules explicitly addressing competency based teaching and assessment. The INSET strategy also aims to address the recruitment, training and career development of Tanzanian primary teachers by providing professional development programmes that bring together pre-service training and continuing professional development so as to

11

Alexander, R. 2008. Education for All, The Quality Imperative and the Problem of Pedagogy. London: DFID. 12

Hardman, F., Abd-Kadir, J., Agg, C. Migwi, J., Ndambuku, J. & Smith, F. 2009. Changing pedagogical practice in Kenyan primary schools: the impact of school-based training. Comparative Education, 45 (1), 65 – 86. 13 Hardman, F., Abd-Kadir, J. & Tibuhinda, A. 2012. Reforming Teacher Education in Tanzania, International

Journals of Educational Development 32, 826 – 834.

20

create a lifelong framework for teachers to upgrade their pedagogic knowledge and skills over a sustained period of time. The development of INSET modules with a focus on pedagogical knowledge supported by in-class coaching, observation and feedback by colleagues in schools, inspectors, education officers and teacher educators is therefore central to the INSET strategy to help support cadres and the teachers they mentor explore their own beliefs and classroom practices and bridge the gap between theory and practice. The evaluation of the school based INSET Pilot Programme therefore had a central focus on what could be observed in the act of teaching and learning. The next section outlines the design and methodology of the study.

21

METHODOLOGY

3.1 Introduction

This section reports on the design and implementation of the research study. Having given an overview of the research design, it goes on to describe the sample and research instruments used, training of data collectors, roles and responsibilities for those conducting the study and the timeline. 3.2 Research questions

The evaluation set out to answer the following research question:

To what extent has the pilot INSET programme impacted on classroom pedagogy and learning achievement?

From this overarching question, it set out to investigate the following sub-questions:

To what extent has the INSET pilot met the objectives of the 2009 INSET strategy in terms of its design, implementation and cost-effectiveness?

What lessons can be learned for the scale up of the national INSET strategy? 3.3 Research design In order to fully address the complexity of the research questions, a multi-method research design using both quantitative and qualitative methods was used. This allowed for methodological triangulation to achieve greater validity and reliability in the study. Each of the observation instruments were closely related to each other to ensure a fully integrated research design with a central focus on classroom processes. They were also designed to be comprehensive, manageable and as low-inference as possible to compare pedagogical practices in Tanzanian primary schools. They were informed by international pedagogical research focusing on what can be observed in the act of teaching (i.e. task, activity, interaction, assessment) so as to ensure the observations of classroom processes are as valid and reliable as is practically possible14. The semi-structured interview schedules for use with stakeholders at the national, regional, district and school level were also designed to elicit their view on the effectiveness and efficiency of the implementation of the INSET and its impact on classroom practices and pupil learning. Available data on the actual costs of implementing the school-based INSET programme compared to college-based provision was also gathered from UNICEF, MoEVT and PMO-

14

Hardman, F. and Abd-Kadir, J. (2010) “Classroom discourse: towards a dialogic pedagogy” in: D. Wyse, R. Andrews & J. Hoffman (eds.) The International Handbook of English, Language and Literacy. London: Routledge, Taylor and Francis, pp. 254 – 264.

22

RALG and analysed to measure cost effectiveness (see Appendix 12). The focus was on calculating both the development and recurrent costs of the programme to inform the Government’s plan for national scale-up to cover all primary school teachers in Tanzania. 3.4 Sample In order to measure the impact of School-based INSET pilot, a replication of the 2008 baseline study of classroom processes was carried out using the same seven District Councils of Bagamoyo, Hai, Magu, Makete, Mtwara(R), Siha and Temeke. In each council, 4 schools participating in the INSET pilot were randomly selected in order to conduct observations and interviews (n=28). In addition to the 28 schools in the INSET intervention group, 4 schools from Kwimba district, next to Magu, where the INSET pilot was not available were selected so as to act as a ‘control’ group for the study15. The overall means for the number of pupils in the intervention and control schools is given in Table 1 below.

Table 1: Class size

Mean Std. Deviation Minimum Maximum

Control 68.19 25.68 33 102 Intervention 69.36 38.20 12 205 All Classes 69.21 36.78 12 205

As the table shows, the means for both intervention and control schools are very similar at just under 70, although the intervention schools show a higher standard deviation due to a wider range of class sizes. 3.5 Research instruments The following research methods were incorporated into the mixed-method design: Systematic observation The classroom interaction study made use of 3 systematic observation schedules that built on the 2008 baseline study and were completed in the natural setting of a lesson. All the instruments were designed to be comprehensive, manageable and as low-inference as possible in capturing current pedagogical practices in Tanzanian primary schools. They were informed by international pedagogical research from 7 countries (Burma, Jordan, Kenya, Kosovo, Nigeria, Tanzania, United Kingdom) focusing on what can be observed in the act of teaching (i.e. task, activity, interaction, assessment) so as to

15

In experimental research, an experimental and control group are used to compare the impact of an intervention, such as the provision of INSET programme. This allows for causal inferences to be made as to the effectiveness of the programme.

23

ensure the observations of classroom processes are as valid and reliable as is practically possible16. They also allowed for a more objective measure of the frequencies and time spent on key teaching and learning behaviours to ensure the validity and reliability of the findings. Because managing the quality of classroom interaction is seen as the single most important factor in improving the quality of teaching and learning, investigating the quality of classroom interaction in Tanzanian primary schools was a central focus of the study. Research into effective teaching has identified the following teaching behaviours as leading to higher gains in learning outcomes17: Lesson clarity Instructional variety Teacher’s effective time management High levels of time on task Maintaining a high success rate for pupils Using and incorporating pupil ideas Strong structuring Appropriate and varied questioning Probing for knowledge (repeated questions to the same pupil) Frequent feedback High expectations of what pupils can achieve Clear and restricted goals. All 12 effective teaching behaviours were incorporated into the design of the three observation instruments to investigate their level of use in the classrooms of those teachers who had participated in the INSET training. Five of the 12 effective teaching behaviours (i.e. instructional variety, using and incorporating pupil ideas, appropriate and varied questioning, probing for knowledge, frequent feedback) focus on classroom interaction because of its centrality to the act of teaching and learning. All three systematic observation instruments drew upon a 3-part teaching exchange structure that is central to teacher-pupil interaction. In its prototypical form a teaching exchange consists of three moves: an initiation, usually in the form of a teacher question, a response in which a pupil, or group of pupils, attempts to answer the question, and a follow-up move, in which the teacher provides some form of feedback (very often in the form of an evaluation) to the pupil's response (from now on referred to as IRF).

16

Hardman, F., Ackers, J., O’Sullivan, M. & Abrishamian, N. (2011). Developing a systematic approach to teacher education in sub-Saharan Africa: emerging lessons from Kenya, Tanzania and Uganda. Compare: A Journal of Comparative and International Education, 41 (5), 669 – 684. Hardman F, Abd-Kadir J, and Tibuhinda A (2012), Reforming Teacher Education in Tanzania, International Journal of Educational Development. 32, 826 - 834 17

Muijs, D & Reynolds (2011) (3rd

ed) Effective teaching: evidence and practice. London: Sage.

24

Research into classroom interaction suggests that the IRF structure can take a variety of forms and functions leading to different levels of pupil participation and engagement, particularly through the use that is made of the follow up move. Such studies suggest that teacher follow up which goes beyond evaluation of the pupil answers, by asking pupils to expand on their thinking, justify or clarify their opinions, or make connections to their own experiences, can extend the answer in order to draw out its significance so as to create a greater equality of participation. The IRF structure can therefore be opened up to create more of a learner-centred form of teaching in whole class, group-based and one-to-one exchanges where teacher questions and pupil responses are woven together into an unfolding exchange to encouraging more pupil-initiated ideas and responses and consequently promoting higher-order thinking. In such cases, the IRF pattern can be said to take on a dialogic function. Observation schedule 1: Frequency of teacher/pupil behaviours: this schedule was designed to capture the frequency of teacher and pupil behaviours occurring during the course of a lesson and was completed at the end building on the time-line analysis (Appendix 2). The observation schedule was informed by pedagogical research into whole class, group-based and one-to-one teaching and focused on what can be observed in the act of teaching (i.e. task, activity, interaction, assessment) so as to collect data on classroom processes that are as valid and reliable as is practically possible. The schedule asked the observer to record the frequency of 26 key teaching and learning activities that are derived from the effective teaching literature using a 4 point scale: 1 = behaviour never observed; 2 = behaviour rarely observed (i.e. once or twice); 3 = behaviour occasionally observed (i.e. four or five times); 4 = behaviour consistently observed. Observation schedule 2: Timeline analysis: this schedule required the observer to record the main teaching activities from a list of prompts every five minute interval in the lesson. In the note section, observers were asked to record activities not covered in the checklist and to provide further contextual information on the activities ticked in the checklist (Appendix 3). Because it is following the time-line of the lesson, the timeline analysis was designed to be formative so as to inform the completion of Observation Schedule 2, which is more summative in design. Observation schedule 3: Computerised systematic observation schedule

The computerised systematic observation schedule first developed in 200318 has been successfully used over a number of years in seven countries as discussed above (Appendix 4). It uses a coding scheme that is fully grounded in the discourse observed (thus increasing the measurement validity of the coding scheme). The coding scheme analyses teacher-pupil interaction by recording the different types, frequency and length of discourse moves made by teachers and pupils.

18

Smith, F. & Hardman, F. 2003. Using Computerised Observation as a Tool for Capturing Classroom Interaction, Educational Studies. 29 (1), 39-47.

25

Interviews At the school level semi-structured interviews were conducted with head teachers (Appendix 5), teachers (Appendix 6), a focus group of 6 Grade 6 pupils (Appendix 7) and parents (Appendix 8) to elicit their opinions on the INSET pilot to better understand their attitudes, perceptions and beliefs with regard to teaching and learning and to qualitatively measure the impact of INSET strategy on classroom practices and pupil learning. At the district level, interviews were conducted with district officers, ward coordinators, ward inspectors and INSET cluster coordinators (Appendix 9). Teacher Colleges were also visited to conduct semi-structured interviews with Principal and college tutors involved in delivering school-based INSET (Appendix 10). Semi-structured interviews were also conducted by Dr Dachi, national consultant to the evaluation and Dean of the School of Education at the University of Dar Es Salam, with a range of stakeholders selected from PMO-RALG, Tanzania Institute of Education (TIE), Agency for the Development of Education Management (ADEM), Institute of Adult Education (IAE), and with Teacher College Principals and Universities to explore their perceptions of the effectiveness of the pilot INSET programme on classroom practices and maximising learning outcomes. Training of data collectors Twenty data collectors were trained and 18 selected from the MOEVT, PMO-RALG, Tanzanian Institute of Education (TIE), IAE, ADEM, Teacher Colleges and Universities based on their knowledge of the pilot INSET programme and experience of conducting classroom observations. Four teams were formed and each team was responsible for visiting two districts over a 2-week period. The data collection training workshop was based in Morogoro and run over the course of three days, including a half-day trialling the observation and interview schedules in schools (Appendix 9). Reliability of coding During the data collection training workshop, inter-rater reliability checks were conducted on Observation schedules 1 and 2 using digitally recorded lessons consisting of descriptive item analysis so as to achieve reliability ratings of at least 90 percent amongst the data collectors19. Analysis of the coding by the data collectors showed that this level was achieved.

19

An 80 percent level of reliability is accepted international as being more than adequate for classroom observation analysis.

26

3.6 Roles and responsibilities International Consultant As international consultant, Dr Frank Hardman, Professor of International Development and Education from the Institute for Effective Education, University of York, UK20, was responsible for:

Overseeing the design and implementation of the evaluation, particularly the focus on classroom processes;

Ensuring that the data collectors are effectively trained to perform their duties;

Supervising the data collection and analysis process;

Taking responsibility for writing the inception and final report of the study in close consultation with the national INSET management team and national consultant;

Organising a dissemination workshop with key education stakeholders to present the findings and recommendations of the evaluation.

National Consultant The national consultant, Dr Hillary Dachi, focused on how the INSET system was working. He therefore oversaw the stakeholder interviews at school and district level, together with interviews with key stakeholders in the University of Dar es Salaam (UDSM); Open University of Tanzania (OUT); Dar Es Salaam University College of Education (DUCE); Institute of Adult Education (IAE); Tanzania Institute of Education (TIE); Agency for the Development of Education Management (ADEM). Interviews were also conducted with the Deputy Permanent Secretary PMO-RALG and the Regional Education Officer Pwani Region (representing the Regional Secretariats-PMO-RALG) and principals of two selected Teacher Colleges. The analysis was done using a descriptive display matrix to show condensed information of each of the respondent/interviewee on a spread sheet. We were thus able to see and compare the results and identify divergences and convergences of views in relation to the following:

1. The INSET delivery system in terms of both professional and technical support 2. Enhancing the delivery system 3. Scaling up the INSET

Data Collectors The 16 trained data collectors worked in 4 teams of 4. Led by a team leader, they were responsible for:

20

The University of York is ranked in the top 10 of UK Universities and in the top hundred of World Universities for its teaching and research.

27

1. Recording background information on the schools, teachers and classes observed. 2. Interviewing district officers, ward inspectors, ward officers, cluster coordinators,

college tutors, head teacher, teachers, parents and pupils. 3. Completing systematic observation of lessons. 4. Supervising the digital recording of lessons. 5. Returning data to UNICEF Tanzania.

3. 7 Timeline for study The study was conducted according to the following timeline shown in Table 2: Table 2: Timeline for study

September 2012 Inception report including work plan, literature search, research design and instruments Identify sample of schools/key informants

October - November 2012 Training of data collectors Data collection and analysis

30 November 2012 Draft report

10 December 2012 13 - 14 December 2012

Final report to MOEVT/PMO-RALG Participatory workshop with key education stakeholders to present findings and recommendations

28

FINDINGS 4.1 Introduction This chapter presents the main findings of the study. Firstly, a detailed analysis of the classroom observation data is presented to investigate the impact of the INSET on classroom practices. This is followed by a detailed analysis of the semi-structured interviews with district education officers, head teachers, teachers, parents and pupils to explore the effectiveness and efficiency of the INSET. 4.2 Analysis of classroom observation data The following section reports on the classroom observation findings from the intervention and control schools and makes comparisons with the 2008 baseline study of teaching and learning practices in Tanzanian primary schools. Frequency of Teaching and Learning Behaviours One hundred and twenty seven live lessons were analysed using a schedule designed to capture the frequency of teacher and student behaviours, most of which have been identified in the international research literature as being effective in raising achievement, occurring during the course of a lesson. A breakdown of the lesson observations by subject and year group in the intervention and control schools is given in Table 3.

Table 3: Breakdown of observed lessons by subject and year group

Grade Subject Control Intervention

Number of schools

Number of observations

Number of Schools

Number of Observations

3 English 4 4 28 28

Maths 4 27

6 English 4 28

Maths 4 28

Total 16 111

Observers were asked to record the frequency of teaching and learning activities observed during the course of the lesson. The 26 observable practices were categorised into 4 sections on the observation schedule. In Section 1 there were 11 teaching and learning behaviours covering lesson clarity, the setting of learning objectives, instructional variety (i.e. use of whole class, paired/group-based and individual teaching) and the general climate of the classroom. Six behaviours were captured in Section 2 covering teacher approaches to questioning, and 6 behaviours were captured in Section 3 covering teacher feedback and follow-up to the questions. Sections 2 and 3 were therefore based on the 3-part, initiation-response-feedback (IRF), questioning exchange as research shows that the IRF exchange is central to the act of teaching and can be used by teachers in different ways to open up or close down pupil participation in the classroom talk. Section 4 was made up of 3 behaviours exploring teacher management of the class.

29

The frequency data were analysed using the Mann-Whitney U test to compare whether there were any statistically significant differences between those participating in the INSET in the intervention schools and those in the control schools who had not been involved in the pilot. It was found that teachers who had participated in the INSET showed significant differences (p < 0.05) in their use of the effective teaching behaviours: checking for prior knowledge, explaining material accurately and clearly, emphasising key points throughout the lesson, creating a positive classroom climate, using paired or group work, changing the classroom to facilitate the learning and using a plenary to summarise, consolidate and extend the learning. Although not significant, teachers in the intervention schools were more likely to state the learning objectives of their lesson, to use a range of teaching and materials and to set homework compared to the non-trained teachers. In Section 2 covering skills in teacher questioning, teachers in the intervention group showed statistically significant differences in their use of open questions and asking pupils to demonstrate their understanding to the class when compared to the control group of teachers. They were also more likely to call on an individual pupil to answer a question rather than cue a choral answer. Similarly, in Section 3 covering the demonstration of skills in feedback, teachers in the intervention schools showed statistically significant differences in their use of probing of a pupil answer, commenting on an answer, building an answer into a subsequent question and encouraging pupils to ask questions. Although not statistically significant, teachers in the intervention schools were more likely to acknowledge a pupil answer and to move around the classroom to provide pupils with feedback. In Section 4 covering skills in classroom management, INSET trained teachers showed significant differences in the way they related to pupils and effectively managed the class and the timing of the lesson. Grade differences When all the Grade 3 lesson observations were compared to all the Grade 6 lessons for each of the 26 teaching and learning behaviours, it was found that in 24 out of the 26 teaching and learning behaviours there was no significant difference between the grades (p < 0.05). This suggests there was little difference in the underlying pedagogic approach used to teach across the two grades. Of the two behaviours that were significantly different (i.e. paired/group work, commenting on pupil answer) they were more likely to be observed in the Grade 6 lessons. However, when each subject was looked at separately to investigate differences between the intervention and control schools the significant differences were the same as those in the general findings. Subject differences However, when the English and mathematics lessons across both grades were compared there were statistically significant differences in 22 of the 26 teacher and learning behaviours (p < 0.05); suggesting teachers of mathematics were far more interactive in their whole class, group based and one-to-one interaction as measured by the effective teaching behaviours than teachers of English. This may have been due to the fact that the mathematics lessons were taught in Kiswahili where teachers and pupils could

30

demonstrate higher levels of communicative competence, thereby allowing for higher levels of verbal interaction. However, when each subject was looked at separately to investigate differences between the intervention and control schools the significant differences were the same as those in the general findings. Timeline Analysis Working independently, two observers were asked to complete the timeline analysis schedule. The schedule required the observers to record the main teaching activities from a list of prompts every five minute interval in the lesson. Observers could record more than one activity in each of the five minute sections of the lesson. Two hundred and fifty four lessons were cross-checked for inter-rater reliability that was found to be above 90 percent and the data merged and analysed. In order to analyse the data a count of the number of times an activity occurred within a 5-minute interval was carried out and divided by the total number of activities in the interval to arrive at a percentage of the time spent on the activity. Officially, Grade 3 and 6 lessons are meant to last 40 minutes so the timeline schedules were completed within these time frames. If the lesson ended before the official designated time, observers coded the behaviour as ‘off-task’. Tables 4 and 5 show a breakdown of the most common teaching and learning activities as a percentage of the lesson time in which they were observed for each of the five minute intervals in the intervention and control schools. Table 4: Percentage of time spent on teaching and learning activities in intervention schools

Intervention

Minutes 1-5 6-10 11-15 16-20

1-25 26-30 31-35 36-40 Overall

A1 18.3 17.7 16.7 17.7 15.2 16.0 16.3 15.0 16.8

A2 13.4 14.3 11.2 9.9 7.7 6.8 5.6 4.2 9.8

A3 16.1 13.0 12.4 11.2 11.7 10.7 9.9 8.7 12.1

A4 4.5 6.1 5.0 3.5 4.0 3.3 3.1 1.9 4.2

A5 5.0 6.3 6.0 6.2 5.5 5.8 3.9 1.9 5.5

A6 6.7 7.3 7.7 7.3 9.1 11.1 13.6 16.7 9.0

A7 2.5 6.6 8.8 9.1 8.8 8.8 7.2 5.9 7.4

A8 4.9 7.7 7.9 7.6 7.7 6.4 6.0 2.8 6.9

A9 5.2 1.6 1.9 2.2 3.5 3.3 2.9 6.4 2.9

A10 0.5 2.1 2.9 4.3 6.2 7.2 9.5 10.4 4.7

A11 7.9 7.1 9.1 9.9 9.7 9.5 10.9 10.6 9.2

A12 14.1 9.6 10.0 8.4 8.2 8.6 8.6 10.4 9.6

A13 0.0 0.0 0.2 0.4 0.4 0.2 0.2 0.2 0.2

A14 0.8 0.7 0.3 2.2 2.4 2.3 2.5 4.9 1.6

100.0

31

Key: A1 = Teacher explanation/question & answer; A2 = Teacher rote/chorus responses; A3 = Teachers writing on chalkboard; A4 = Teacher reading to whole class; A5 = Pupil reading to whole class; A6 = Pupils working individually; A7 = Pupils working in pairs/groups; A8 = Pupil demonstrating to class; A9 = Teacher reviews lesson topic; A10 = Teacher marking work; A11 = Class Management; A12 = Class administration; A13 = Interruption to lesson; A14 = Pupils off-task Table 5: Percentage of time spent on teaching and learning activities in control schools

Control

Minutes 1-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 Overall

A1 21.3 21.4 18.2 15.2 14.5 12.7 13.8 14.1 16.7

A2 13.3 12.9 10.4 4.5 4.3 7.0 1.5 2.8 7.7

A3 17.3 21.4 18.2 13.6 11.6 14.1 12.3 7.0 15.5

A4 6.7 5.7 6.5 4.5 5.8 4.2 3.1 1.4 5.2

A5 4.0 10.0 5.2 6.1 1.4 4.2 3.1 0.0 4.9

A6 1.3 4.3 6.5 10.6 14.5 12.7 16.9 21.1 9.5

A7 4.0 5.7 3.9 6.1 7.2 4.2 1.5 1.4 4.7

A8 2.7 5.7 3.9 3.0 1.4 1.4 0.0 1.4 2.6

A9 0.0 0.0 1.3 3.0 1.4 4.2 1.5 4.2 1.6

A10 1.3 0.0 1.3 1.5 8.7 8.5 13.8 16.9 5.0

A11 13.3 8.6 13.0 21.2 14.5 9.9 13.8 12.7 13.5

A12 12.0 1.4 5.2 3.0 4.3 7.0 4.6 4.2 5.4

A13 1.3 0.0 1.3 1.5 1.4 1.4 1.5 0.0 1.2

A14 1.3 2.9 5.2 6.1 8.7 8.5 12.3 12.7 6.4

100.0

An independent-samples t-test was used to statistically analyse differences in the time-line distribution of teaching and learning activities between the intervention and control schools. It was found that, as in the frequency observations, teachers who had participated in the INSET training were showing significant differences in the use and time spent on paired/group work, pupil demonstration of their understanding to the class, class management, interruptions to lessons and pupils off-task (see Figures 1 and 2). In other words, teachers in the intervention schools were significantly more likely to involve the pupils in group work and pupil demonstration, and to spend less time on class management issues. There were also fewer interruptions to their lessons and pupils were more likely to be on-task. It is also interesting to note that intervention schools were spending more time on class administration (9.6 percent of the time) compared to control schools (5.4 percent of the time). This suggests teachers who had participated in the INSET were spending more time on organising transitions in their lessons (e.g. moving from whole class to group-based activities, giving out teaching and learning materials) because of more varied teaching and learning approaches. As Figures 1 also shows, teaching and learning activities were more evenly spread across the lessons of intervention teachers, suggesting higher levels of teacher-pupil interaction.

32

Key: A1 = Teacher explanation/question & answer; A3 = Teachers writing on chalkboard; A7 = Pupils working in pairs/groups; A8 = Pupil demonstrating to class; A11 = Class Management; A13 = Interruption to lesson; A14 = Pupils off-task The timeline findings from the control schools more closely reflect those from the 2008 baseline study where overall across the teaching of English and mathematics it was found that teacher directed activities (explaining, question and answer, teacher rote and chorus response, writing on the chalk board, reading to the class, asking pupils to read, individual seat work, lesson review, teacher marking, class management, administration) took up over 84 percent of the lesson time. More ‘learner-centred’ forms of learning (i.e. paired or group work, pupil demonstration) accounted for just 7.3 percent of the lesson time, similar to the 2008 baseline, compared with 14.3 in the intervention schools. Non-curricular activities (i.e. interruptions, off-task) took up a further 7.6 percent of the time in the control schools compared to 1.8 percent in the intervention lessons. Most of the time off-task in the control occurred in the remaining ten minutes of the lessons in the suggesting they were more likely to end early. Although not significant, intervention schools spent more time reviewing lessons compared to control schools, 2.9 and 1.6 percent of the time respectively.

0

5

10

15

20

25

Min 1-5 Min 6-10 Min 11-15

Min 16-20

Min 21-25

Min 26-30

Min 31-35

Min 36-40

Pe

rce

nta

ge o

f ti

me

sp

en

t o

n a

ctiv

itie

s Figure 1: Intervention Group Timeline Analysis

A1

A3

A7

A8

A11

A13

A14

33

Key: A1 = Teacher explanation/question & answer; A3 = Teachers writing on chalkboard; A7 = Pupils working in pairs/groups; A8 = Pupil demonstrating to class; A11 = Class Management; A13 = Interruption to lesson; A14 = Pupils off-task Analysis of Digitally Recorded Lessons The computerised interaction analysis schedule analysed teacher-pupil interaction by recording the different types of discourse moves made by teachers and pupils. The coding system primarily focused on the initiation-response-feedback (IRF) structure by gathering data on the types of teacher questions, whether questions were answered (and by whom), and the types of follow-up given in response to answers. It also recorded the number of pupil initiations in the form of questions. Responses were coded according to whether a boy or girl answered, or whether there was a choral reply. Teacher follow-up to a pupil response was coded according to whether it was affirmed, praised or elaborated upon. A Grade 3 and 6 English lesson from one school from each of the eight districts was randomly selected and intensively analysed using the computerised systematic observation schedule to record the different types, frequency and length of discourse moves made by teachers and pupils. English lessons were chosen to save time on lesson translation back in the UK where the computerised analysis was conducted. Across all of the schools, lessons were timetabled to last 40 minutes. However, the average lesson of the recorded lessons in the sample was 35 minutes in length. Most of the lessons were mainly taught through the medium of English, although code switching to Kiswahili and mother tongue was used to varying degrees in over a quarter of the classrooms, mainly at Grade 3. The average class size was 62 pupils. The use of IRF moves in whole class teaching by intervention and control schools was statistically compared to investigate whether there were any differences in patterns of whole class interaction using an independent-samples t-test The following differences between trained and non-trained INSET teachers were found to be significant at p<0.05:

0

5

10

15

20

25

Min 1-5 Min 6-10 Min 11-15

Min 16-20

Min 21-25

Min 26-30

Min 31-35

Min 36-40

Pe

rce

nta

ge o

f ti

me

sp

en

t o

n a

ctiv

itie

s Figure 2: Control Group Timeline Analysis

A1

A3

A7

A8

A11

A13

A14

34

teacher informs, teacher questions, cued elicitations, teacher check, teacher direct, pupil questions, girl answers, choral answer, probe, comments, praise and no follow-up. As in the frequency and timeline analysis, the intervention schools generally had a more interactive style in terms of their use of teacher questions, and participation strategies (cued elicitations and teacher checks). INSET trained teachers also used more teacher explanation and gave more praise in their feedback, encouraged more pupil questions and individual answers from girls and boys, and probed and offered more comments on pupil answers (i.e. rephrasing, building or elaborating upon an answer). They also used more teacher direction mainly due to the greater use of group/paired work which necessitated the giving of directions in regrouping the class and setting up activities. In analysing the use of paired or group work (i.e. when pupils were observed interacting together and exchanging ideas through exploratory talk), it was found that 63 percent of the INSET trained teachers used some form of peer interaction in their lesson compared to just 6 percent of the non-trained teachers. A similar picture emerged with the use of open-ended questions (i.e. questions eliciting a range of responses): INSET trained teachers were twice more likely to ask an open question: 11 percent of the questions asked by INSET trained teachers were open compared to 3 percent asked by non-trained teachers. The video analysis also showed that a greater range of organisational arrangements were being deployed by intervention schools to meet different educational goals. In the 2008 baseline, 98 percent of classrooms were organised using a traditional classroom layout (i.e. desks organised in rows) similar to the control schools; this compares to 64 percent of the intervention schools using an alternative classroom layout reflecting the higher use of group work. 4.3 Analysis of interviews In the districts and school where the MOEVT INSET was piloted interviews were conducted with District Education Officers, Head teachers, Teachers, Parents and Pupils. In the control schools, interviews were only conducted with the pupils to explore their perceptions of the range of teaching and learning activities currently taking place in the classrooms of teachers who had not received the INSET training. The semi-structured interviews were analysed using a content analysis approach in which research themes were generated from the data so as to create categories into which the interview responses could be assigned. Teachers One hundred and seventeen teachers who had participated in the INSET programme were interviewed to elicit their views on the effectiveness of the programme. When asked about their current level of motivation for teaching, 33 percent of the teachers stated they were ‘very motivated’ and 55 percent indicated ‘on the whole motivated, with just 12 percent stating that they were ‘not at all motivated’.

35

In response to a question about which aspects of the training they had found the most useful, almost three quarters of the teachers stated they found the study modules very useful with a third specifically referring to the pedagogy module. Sixty percent of the teachers found the cluster group meetings to be very effective and 40 percent found the classroom observations to be very useful. When asked about how the INSET programme had improved their teaching, 70 percent of the teachers felt that it had improved pupil participation in lessons and broadened their teaching repertoire through a greater use of paired and group and teaching aids drawn from the local environment. Over a half of the teachers interviewed also felt that the training had improved lesson planning and helped teachers to identify the learning needs of the pupils they teach. Similarly, in response to the question on how the training had improved pupil learning, over three quarters of the teachers stated it had increased the active participation of pupils and improved their confidence and motivation. As one teacher from Nkondo stated: ‘Pupils enjoy studying in groups and having a chairperson and secretary. They are participating and they are no longer afraid.’ When asked about observations of their teaching as part of the follow up to the INSET training, 70 percent stated they had been observed. When probed about who had observed them, a third of the teachers indicated they had been observed by a peer, 24 percent by a head teacher, 18 percent by a college tutor and 17 percent by a ward education coordinator. It was interesting to note that in Mtwara, all 21 teachers had been observed and college tutors had participated in 16 of the teacher observations whereas in Hai, Makete, Siha and Temeke none of the teachers had been observed by a college tutor. In response to the question about when teachers undertake most of their training, over 60 percent of the teachers interviewed stated that they studied in the evenings after school with 30 percent meeting in study groups. Nearly a third of the teachers said they studied during school hours in breaks and at lunch time and a similar number stated they studied at weekends. Overall, the INSET trained teachers were very positive about the training they had received and thought it should be scaled up at the school and national level to include all teachers. They thought more time should be made available for school-based INSET through the allocation of training days and study time during the working week. They also indicated that they wanted the training to be accredited and rewarded through promotion and for there to be more face-to-face training, cluster support and follow up in the classroom. However, 30 percent of teachers raised concerns about the timing of feedback on assignments. As one teacher from Mtwara noted: ‘There was no feedback on assignments submitted to teacher colleges for marking and teachers don’t know what they have achieved or what went wrong’.

36

Twenty percent of the teachers found the English module more difficult to study than the pedagogy and mathematics module. This may have been due to the fact that it the module was written in English whereas the pedagogy and mathematics were in Kiswahili. Head teachers Twenty eight head teachers were interviewed for the seven intervention districts. Overall the response was very positive towards the pilot INSET. There was a general understanding that INSET had impacted on teacher and pupil performance, creating a more learner-centred form of teaching, greater confidence in teachers and students alike, more involvement of students in their learning and greater subject matter competence from teachers and pupils. Mentoring, cluster groups, observations and reading of the modules by the teachers, were deemed to be the most important features of the programme. As one head teacher from Mtwara stated: ‘Sharing and exchanging information among teachers through small groups and cluster meetings has added value to teaching’. Most teachers were reported to do the INSET work after school hours (often timetabled by the school so as part of the school day but after class) but also during private time on weekends and evenings. Twenty two of the head teachers reported their teachers had been observed, with peer review playing a major role. Some head teachers indicated, however, that there was a need for more external input into the observation of teachers, particularly from college tutors. Twenty five of the head teachers also stated that their trained INSET teachers had been involved in the delivery of school-based training for non-INSET trained teachers that drew upon the modules. Some head teachers indicated that they would have liked additional training in leadership on the school-based INSET. In terms of comments, the need for an INSET budget and materials for teachers was a constant theme. Other themes included other logistical and infrastructure problems, such as difficulty of transport to allow teachers to attend cluster meetings due to time, distance and cost, as well as a need for desks, water, sanitation and housing for teachers. Some modules were seen as too difficult, particularly with respect to the English, for which some requested additional training. While 25 of the head teachers reported they had a whole school development plan, 7 did not feature INSET as a priority and those that did, many head teachers reported they could not implement the plans due to a lack of an INSET budget. There were additional comments that the ‘train the trainer approach’ was not ideal and that each teacher should receive training. Perhaps the most significant comment from a programmatic point of view from all of the head teachers was that relating to the need for providing incentives to teachers, such as certificates, additional pay and promotion for those who participate in the INSET, a view summed up by a head teacher from Siha:

37

‘the INSET programme needs to be tied with certification, accreditation and salary increment.’ School Committees Twenty eight focus group discussions were carried out with representatives from the School Committees. Twenty one of the School Committees stated that they were aware of the INSET and 14 responded that they had been involved in discussions of the INSET programme. While 26 School Committees stated they had been involved in drawing up the school development plan, 21 reported that INSET was not part of the plan. Therefore most School Committees knew about the INSET but only in a general sense, with most aware that that the programme had to do with teacher training and many knew that it involved teachers working together in the school and at cluster meetings. While not familiar with the technical detail of how the INSET was being delivered in their schools, most of the School Committees (24 out of 28) agreed that the programme had improved the quality of teaching in their schools through improved teacher-pupil relations and higher levels of commitment from the teachers. Similarly, 26 School Committees reported that the quality of education in their schools had improved with more active pupil participation in lessons and with pupils reporting they liked school more. Overall, many of the School Committees focused more on budgets and infrastructure in their discussions and not as much on teacher or school performance. Some School Committee members would have liked more information on the INSET programme, to have been more involved in discussions of its implementation, and to have received orientation training. They also expressed general concerns about the lack of teaching learning resources, poor infrastructure and the need for all teachers to be given INSET training. Pupils Focus group interviews with 4 Grade 6 (2 boys/2 girls) pupils were conducted in each of the 28 intervention schools to elicit their views on the impact of the School-based INSET on teaching and learning practices. Focus group interviews were also conducted with Grade 6 pupils in the 4 control schools in the Kwimba District for comparison with the INSET intervention schools. When asked about which subjects they liked the most, pupils from all 8 districts reported that they liked mathematics, English and science the most. Pupils in the intervention schools were asked about whether they had noticed any changes in the way their teachers taught since the previous year. There was general agreement that teaching practices had changed, particularly in the use of group work, teaching and learning aids and pupil participation in lessons. Many reported that teachers were friendlier, that they had more time for the pupils and they allowed pupils to ask questions. They also reported that lessons were more varied and that group work was a daily feature of their lessons, particularly in mathematics, English and science. This was in contrast to the focus

38

group interviews conducted in the control schools in Kwimba District. Two of the focus groups reported group work was rarely used and 2 reported that it was never used. When asked what one thing would help them become better learners, 3 of the 4 focus groups from the Kwimba district stated that ‘better teaching’ would help them learn. Teacher Training College Principals and tutors Three colleges were visited to interview the principals and 6 college tutors. They reported that the Teacher Colleges (TCs) had been responsible for administering the INSET programme, liaising with the District Councils and for mentoring, monitoring and assessing the teachers. They were generally positive about the design of the programme and the modules and thought it had improved teacher effectiveness and pupil learning. They also thought the programme was time efficient as teachers could study the modules in their own workplace supported by school and cluster-based meetings with other teachers and follow up in the classroom. The main constraints they identified were the lack of an INSET budget to cover travel costs to the schools and clusters, thereby limiting the number of visits they could make to mentor, monitor and assess the teachers and to liaise with the District Councils. Overall, the principals and college tutors thought the programme should be extended to include all teachers and school subjects, and for there to be academic accreditation of the programmes with official study time being made available to teachers throughout the school year. They also felt there was a need for more training of teacher educators, including ward education officers, district INSET and cluster coordinators. District Education Officers At the district level, 5 District Education Officers, 7 District Inspectors, 5 District INSET Coordinators, 24 Ward Education Officers and 4 cluster coordinators were interviewed about the effectiveness of the MOEVT school-based INSET programme. There was general agreement that the training had improved teacher competence, subject knowledge and confidence leading to more learner-centred approaches. There was a similar level of agreement that the modules, particularly the pedagogy module, had been effective in training the teachers in their own schools and that the mentoring, observation, school and cluster meetings had also provided a good level of support. The training was also viewed as being cost effective because the distance learning materials ensured teachers were not taken away from their schools and they could study in their own time during the school day and in the evenings and at weekends. When asked about their own role in the pilot INSET programme in their district and ward, the ward education coordinators, district INSET coordinators, cluster coordinators and inspectors indicated that they had mainly been involved in the mentoring and observation of teachers, and in monitoring the programme, attending training sessions, administering the programme and liaising with the schools, clusters and colleges. School inspectors also reported they had noticed improvements in the quality of teaching and learning in their visits to schools. In response to the question on INSET budgets, the majority of education officers were not aware of its existence. Not surprisingly, in

39

response to the question asking for further comments, the lack of an INSET budget being made available at the district, ward and school level was seen as a major challenge for the sustainability of the programme which they felt should be rolled out to all teachers. There was also general agreement that the programme needed to be accredited and that teachers who took part in the INSET needed to be rewarded with higher pay and promotion. They also felt that there was a need for further training, accreditation and career advancement of the teacher educators working at the school level to help build their capacity for delivering and monitoring school-based INSET. National stakeholders A total of 13 stakeholders were interviewed to seek their views about the INSET delivery system in terms of both professional and technical support and their opinion on how the delivery system should be enhanced and the INSET scale up. The majority of the respondents were aware of the advantages of the school based INSET at scale using the distance learning approach and its core activities of mentorship, self-study and face to face sessions. Teacher Colleges In addition to the 3 Teacher College (TC) principals and 6 college tutors visited by the data collection teams, one other college principal and INSET coordinator were asked to comment as to whether the colleges and tutors were able to provide professional support to the teachers as required, they stated categorically that tutors managed to do face to face tutorials by visiting the clusters and teachers in the schools only once. The coordinator was of the view that it is for this reason a number of teachers were complaining about several units in English and mathematics modules being too difficult for them to comprehend and apply on the local contexts. The emerging general picture is that principals and coordinators consider that several face to face sessions or intensive face to face sessions should have been designed in the INSET programme. Another question that we asked was directed at technical support; in particular If TCs received funds to implement the INSET activities in the clusters and schools. The general response was that they received funds for training of expert teachers including coordinators at cluster and school levels; distribution of modules in the schools; and face to face tutorials. The challenge was that the funds were not enough to meet all needs such as monitoring of the progress of INSET in the clusters and schools, as well as assisting learners to effectively interact with the modules.

One critical issues that emerged is that teachers had finished reading the modules and working on the assignments which subsequently were submitted with the portfolios to the TCs to be marked but have not received any feedback from the tutors. Answers from a follow up question revealed that tutors have not been able to provide any feed back to the teachers because of lack of funds to dispatch the marked portfolios back to the clusters and schools.

40

Institute of Adult Education It was also stated that the Teacher Resource Centres (TRCs) were dysfunctional and their professional support activities so diminutive that they are insignificant, a finding that is supported by the 2008 baseline on INSET provision. The general consensus was that TCs have proved their appropriateness as institutions for professionally backing up the INSET programme. It was recalled that the 1995 Tanzania Education and Training Policy mentions school clustering as another option for effective continuous professional development programmes because of their proximity to the schools and the fact that are formed and managed by teachers themselves. Technically this supports the INSET delivery design which recognises the importance of clusters in fostering consultations among teachers, collaboration in the use and application of distance learning materials, learning cycles and dialogue. It was argued that professionalism in teaching is no longer an individual competence, but includes the ability to function and the capacity and willingness to share experiences that can enhance the quality of teaching and learning. PMO-RALG Conversely, one caveat was explained by a respondent from PMO-RALG that an expanded Tanzania education system has little capacity to facilitate frequent, systematic and regular meetings and exchanges of scholarly materials, seminars and study workshops and colloquia in the clusters. It was stated further that teachers need to own the process, rather than being pawns waiting for their manner of conduct to be determined by people and forces outside their realm and that INSET should not be conceived as a top down innovation but a participatory one. Arguably, teachers usually would attach a high premium to the initiatives in which they have a stake. Devolving the ownership of the INSET to the wards clusters and schools should be a core concept in the INSET professional and technical support. Cluster activities should not be conceived cost-free and simple to run and manage. The INSET programme is creating an environment for extra work loads and increased opportunity costs. Without motivation, incentives and support teachers engagement with the programme will be minimal which would in turn compromise the objectives of the INSET programme. The education system governance practices suggest an eclectic model in terms of functions that are centralized and decentralized guided by the Decentralization by Devolution (D by D) policy which was translated into a structural and institutional reform, by which the Central Government has devolved powers, responsibilities and resources to Local Government Authorities (LGAs). One of the precepts of D by D is fiscal decentralization which is the reassignment of revenue raising powers and expenditure from the centre to the LGAs, and using their own financial resources in their areas of jurisdiction. In this regard, we asked a member of the Regional Secretariat (PMO-RALG) as to whether the LGAs had supported the INSET by funding any of its activities. The answer was that the councils were advised to budget for the INSET programme as a component of capacity building. However, this was not followed up by a government circular for it to be strictly enforced. It was confirmed that the INSET programme was not allocated any funds for 2012/13 fiscal year and the officials were sceptical if it would be allocated funds in the 2013/14 fiscal year.

41

We asked for an additional explanation for this assertion. It was argued that capacity building is a synonym for short term and long term training which are budgeted under the other charges of the recurrent budget. Every year the Government sets a ceiling on sub-votes as defined by priorities set in the Medium Term Expenditure Framework. In this case the budget was pegged on the 2010/11 fiscal year framework which had not allocated any funds for INSET. The Ministry of Finance has the obligation to provide conditional subventions and block grants to the LGAs for social services and capital investments. One official from PMO-RALG observed that, ‘... Interventions that require implementation by LGAs have to also commit and guarantee availability of financial resources...or debunk where the resources are going to come from... The LGAs’ income from taxation and other fiscal instruments has not been sufficient enough to meet expenditure obligations that are not dependent on Central Government subventions. Available statistics show that the majority of Councils depend on the Central Government for up to 90% of their recurrent and development expenditures...’ It was recommended that the INSET modules could be adapted by TCs and used as core support or supplementary materials for PRESET, available to every student teacher free of charge or at a small cost built into tuition fees. Arguably, for many new teachers the only influence on their teaching in the field is their own experience of classroom learning and teaching practice. They are not assured of finding professional support or best practices in the performances of experienced teachers when they begin teaching. In this way new teachers will have a fund of knowledge and reference or back up materials for subject content knowledge and ‘constructivist’ pedagogic practices when they begin teaching. A respondent pointed out that one advantage of supplying to and using INSET materials in the TCs is that the modules use will be scaled up as teachers are posted to teach in different regions, districts and schools. Agency for the Development of Education Management Others commented that the demand put on the councils to deliver the outputs of government and donor partners in most cases does not commensurate with their capacity both in terms of staff and resources. Whilst the capacities of individual staff and departments at the councils level are variable, the general picture is that the number of interventions that have to be implemented and monitored stretches the already ‘over stretched staff’. It was suggested that other institutions and organisations which have capacity to provide technical support to the INSET programme should be identified and brought on board. It was pointed out for instance that within the institutional arrangements for the INSET delivery the structure or institution that provides a professional backup to the TCs and tutors is blurred. There are a number of views that are emerging from the analysis in relation to the cost efficient ways of rolling out the INSET programme. However, these views are converging

42

on one point that it should be rolled out gradually in a phased manner over a specified period of time.

(a) PMO-RALG

The INSET in the pilot districts should be scaled up to cover all wards, clusters and schools. Then these would be model districts from which other districts in the respective regions will learn from success stories, best practices and experiences.

(b) Tanzania Institute of Education

One district from each of the rest of regions should be selected on the basis of mutually agreed criteria to become a model district for other districts. A few wards, clusters and schools should be selected and supported to become centres of excellence in terms of cluster coordination, expert teachers, peer learning and networking. University of Dar es Salaam The mentioned advantages of the phased roll out approach were that available resources are not spread thinly and it allows for time to build district capacity to: (a) implement the intervention: (b) mainstream it in the development plans and budgets; and, (c) develop an effective communication strategy for informing key stakeholders about the intervention and their roles and responsibilities in its implementation and lines of decision making and accountability. Open University of Tanzania The coordinator of Teacher Education in Sub-Saharan Africa (TESSA) at the Open University of Tanzania was of the view that modules should in addition to their current print form, be made open and freely available in a variety of electronic formats such as compact discs and flash discs (data sticks). Dar es Salaam University College of Education Another respondent was of the view that the scaling up of the INSET should be supported by an electronic platform such as websites (internet), television and radio broadcasts. Overall findings The overall findings from the 3 observation instruments and stakeholder interviews triangulate well and suggest the INSET pilot has been well received at the district and school level and school-based INSET is having a significant impact on teaching and learning practices in line with the international literature on effective teaching and therefore fit for purpose. In the 2008 baseline it was found that Tanzanian primary teachers mainly used a transmission model of teaching in which the teacher often used a chalk board and/or textbook to transmit recipe knowledge for recall. There was also little difference in the way mathematics and English were being taught across Grades 3 and 6.

43

Pupils spent a great deal of time listening to the teacher explaining, asking questions, writing on the chalk board, reading and managing the class. The closed nature of the questioning and direction by the teacher meant that pupils were rarely given the opportunity to ask questions or contribute their ideas. It therefore limited the extent to which pupils could develop their oral skills and critical thinking, and take responsibility for their own learning. The findings from the 2008 baseline were replicated in the 4 control schools in Kwimba. However, the findings from the frequency and timeline analysis and stakeholder interviews suggest teachers who had undergone the INSET training showed significant shifts in their beliefs and pedagogic behaviours compared to the baseline and control schools. They were using more dialogic forms of engagements when interacting with pupils in whole class, group based and one-to-one situations and providing for a higher level of collaborative learning.

44

CONCLUSIONS Overall, the findings from the evaluation of the pilot MOEVT INSET support the view that bringing about changes in teachers’ beliefs, knowledge, understandings, skills, and commitments, is possible through a well designed school-based INSET system and programme that takes into consideration the contextual reality in which teachers work. The findings also support the view that improvements in pedagogical practices requires professional development programmes that upgrade pedagogic knowledge and skills over a sustained period of time rather than through disjointed one-off courses. They also show that Tanzanian teachers need to be provided with opportunities to work together at the school level so as to learn from one another through mentoring or peer coaching and by conducting whole school training to collectively guide curriculum, assessment and professional learning decisions. The findings of the evaluation support the view that it is best to focus on the school as the most effective level of intervention to improve the quality of teaching and learning by involving the school head teacher and all the teachers in creating a genuine learning community through ownership of the process. The initial costing of the school-based INSET (Appendix 12) suggests the distance learning programme is providing value for money compared to traditional college-based PRESET and INSET provision. The figures show that college-based courses are ten times the cost of school-based training and that as the system beds in costs are likely to fall. Overall, the findings from the INSET evaluation suggest that enhancing the capacity of the teaching profession is crucial if the quality of teaching and learning in primary schools in Tanzania is to improve. By focusing on the classroom, it shows that school-based training can help teachers develop more of a dialogic pedagogy to broaden the repertoire of whole class teaching currently found in Tanzanian classrooms. In this way dialogue and discussion can be included alongside the more traditional drilling, closed questioning and telling, thereby raising cognitive engagement and understanding. Such an approach builds on the traditional model of whole class teaching which is found in many Tanzanian primary classrooms but avoids the simplistic polarisation of pedagogy into ‘teacher-centred’ versus ‘child-centred’ that has characterised much of the educational discourse in the international donor community. It will also help to ensure there is a better balance and blending of local cultural practices in Tanzania with internationally informed teacher education reforms. In terms of changing interactional and discourse practices the findings of the evaluation support the view that training teacher in alternative approaches to teacher-led rote and recitation can make a difference. The school-based English, mathematics and pedagogy modules provided training in the use of both open and closed questions, sharing questions at the start of a lesson, encouraging pupils to ask questions, asking pupils in pairs to discuss a question for a minute before they answer, getting a pair or group of pupils to set questions for another pair, or beginning a lesson by giving pairs or groups a question to answer from the last lesson. Similarly, in following up a pupil answer, teachers were trained to use effective alternatives to simply giving direct feedback; these included probing, commenting on a response to exemplify, expand, justify or add additional

45

information, and building a pupil response into a subsequent question, thereby acknowledging its importance to the classroom discussion. The findings of the evaluation suggest the training is proving effective and many would like to see it scaled up and sustained over time. In order to bring about such pedagogic changes as found in the current study, the evaluation supports the view that the school-based teacher development programmes needs to continue with helping teachers to explore their own beliefs and by getting them to reflect on their classroom discourse practices as a way of enhancing expert thinking and problem solving so as to bridge the gap between theories and actual classroom practice. Therefore, in the context of Tanzania, a national model of INSET training which builds on existing systems and structures at school and school cluster level should be adopted as the most effective way of providing support and development to teachers. The findings also show that working at the school and cluster level will also help to ensure that teacher education is part of a broader capacity development strategy that supports all actors in the education system, including, for example, head teachers, district education officers and teacher trainers, and that it is cost effective against all the other competing demands in a resource-poor environment like Tanzania. While the evaluation findings support the view teacher education and professional learning should be largely located in the school environment, it should also be recognised that such provision requires an investment of time and money in building partnerships, collaboration and delegation. The capacity and training needs of those charged with organising and providing the training, mentoring and coaching, such as district officers and college tutors, ward education officers and inspectors, will be central to the success of school-based training, as will the creation of incentives and accreditation for those college tutors and teacher mentors who will be working with teachers in school and for those teachers who complete the INSET training. It also requires a clear division of roles and responsibilities between national, regional and district offices, and between head teachers, schools, tutors and mentor colleges. This evaluation study does not conclude that school-based INSET training alone can address the problems faced by teachers and learners in Tanzanian classrooms where resources are scarce and the education system is likely to continue to face many constraints. Rather, it suggests that a holistic approach to building an education system is required which recognises the socio-cultural context in which teachers work and emphasises capacity building and the equitable distribution of resources at national, regional, district and school level. However, the quality of the teachers’ expertise is essential to raising standards in the country’s primary schools, and the evaluation of the INSET pilot suggests that systematic school-based training, together with management and career structures that result in consistent and high quality performance by teachers, will contribute to teacher professionalism and classroom practice, and significantly raise educational achievement in Tanzania.

46

EMERGING LESSONS Overall, the evaluation of the INSET pilot supports the view that enhancing the capacity of the teaching profession is crucial if the quality of teaching and learning in Tanzanian primary schools in the region is to improve. The main lessons emerging from the evaluation are:

1. A move away from ad-hoc provision, which is inequitable and can be of variable quality, to a more systematic, longer-term and sustainable approach where the teacher is much more involved in his or her on-going professional development, and where other actors play critical supporting roles, will do much to enhance the capacity of Tanzanian primary teachers to deliver quality education.

2. A multi-mode system, including distance learning and teacher development at school and school cluster level, is the most cost effective way for ensuring national INSET coverage.

3. School-based provision requires a partnerships, collaboration and delegation. It also requires a clear division of roles and responsibilities between national, regional and district offices, and between head teachers, schools, tutors and mentor colleges.

4. There needs to be clear linkages between PRESET and INSET and an alignment of policies, plans and institutional arrangements for teacher education, so that an effective school-based programme can ultimately be implemented at a national scale.

5. A review of the funding of PRESET and INSET is needed to bring about a better balance between the time and money spent on initial education and training and subsequent teacher professional development. In the long term it may make sense to shorten the period of PRESET in favour of more school-based INSET, thereby directing more training resources towards those teaching in the classroom.

6. Capacity building and incentives also need to be devolved to those responsible for delivering school-based INSET. This will ensure that teacher education is part of a broader capacity development strategy that supports all actors in the education system, including, head teachers, district education officers and teacher trainers, and that it is cost effective against all the other competing demands on the education budget.

7. In decentralizing teacher education in Tanzania it will also necessary to encourage transparency about the budget, build capacity at all levels of the system, and to consult all stakeholders on the distribution of responsibilities, resources and incentives.

47

8. Putting in place a systematic monitoring and evaluation system with input from

stakeholders across all levels of the education system will help improve accountability, planning and implementation, and assist in knowledge sharing.

This evaluation study does not conclude that school-based INSET training alone can address the problems faced by teachers and learners in Tanzanian classrooms where resources are often scarce and the education system is likely to continue to face many constraints. Rather, it suggests that a holistic approach to building an education system is required which recognises the realities of the context in which teachers work and emphasises capacity building and the equitable distribution of resources at national, regional, district and school level. However, the quality of the teachers’ expertise is essential to raising standards in the country’s primary schools, and the evaluation of the INSET pilot suggests that systematic school-based training, together with management and career structures that result in consistent and high quality performance by teachers, will contribute to teacher professionalism and classroom practice, and significantly raise educational achievement in Tanzania.

48

APPENDIX 1: TERMS OF REFERENCE Appendix 1: Terms of Reference The United Nations Children Fund (UNICEF) Tanzania, is seeking the interest of qualified and highly experienced consultants to provide Technical Assistance to the Ministry of Education and Vocational Training (MoEVT) in Tanzania for the Evaluation of the pilot phase of the School Based In-Service Education and Training (INSET) Program for Primary School Teachers TERMS OF REFERENCE

Position Titles:

Consultants, Evaluation of School Based INSET pilot programme

Fee: Negotiable

Location: Dar es Salaam with travel to 7 Pilot Districts (Bagamoyo, Hai, Magu, Makete, Mtwara, Siha and Temeke)

Duration: 40 working days between 06 August 2012 – 09 November 2012

Reporting to: Director of Teacher Education Department, MoEVT and; Education Specialist, UNICEF.

Background and Rationale In its commitment to: raise the quality of basic education and; achieve the Dakar Framework on Education for All and the Millennium Development Goals, the Government of Tanzania has embarked on major reforms. The Government developed and implemented the Primary Education Development Plan (PEDP: 2002-2006) and the Secondary Education Development Plan (2004-2009). During the implementation of these plans it was further realized that teacher education needed to be strengthened so as to respond positively to the demands of the PEDP and SEDP plans. Thus, the Teacher Development and Management Strategy (TDMS 2008 - 2013) was formulated and launched in 2008. The goal of the TDMS is: ‘to have and sustain adequate numbers of competent teachers and tutors to effectively support the pre-primary, primary, secondary, adult and non-formal education, as well as Teachers’ Colleges’ (URT, 2008). TDMS is set to improve the quality of Pre-Service Teacher Education (PRESET) and In-Service Teacher Education and Training (INSET) as an important step in the Government’s efforts to provide a quality basic education for all. Key to delivery of a quality basic education for all is the availability of well trained, competent, motivated, adequately rewarded and well supported teachers. INSET was identified as an area of particular concern as current INSET provision is uncoordinated, ad-hoc and of varying quality. The Government decided to concentrate its efforts on developing a quality, sustainable, cost-effective INSET system for primary school teachers. In 2009 the Ministry of Education and Vocational Training (MoEVT) approved the INSET strategy for primary schools teachers which also included an M&E framework for

49

implementation of the strategy. The strategy was approved following a highly consultative process which involved Ministries, Departments and Agencies; Development Partners (DPs) and Civil Society Organisations (CSOs). It was developed following a baseline study conducted to assess the quality of teaching and learning processes actually taking place in the classroom. The study revealed that; classroom teaching was predominantly teacher-led and based on ‘chalk and talk’ methods; the majority of teachers had not had appropriate INSET opportunities to improve the quality of their interactive teaching and; teachers faced difficulties in the preparation of teaching and learning materials, managing larger class sizes and catering for pupils with varied learning needs. Teachers’ mastery of key subjects like science, mathematics and English were of particular concern. Generally, the existing teaching and learning conditions did not easily support learner centred methods contrary to the current competency-based curriculum and assessment. Thus, the INSET strategy was necessary to address the existing gaps. Against this background the Government, supported by UNICEF, began piloting a new school based INSET model based on the INSET strategy in seven Councils (Bagamoyo, Hai, Magu, Makete, Mtwara(R), Siha and Temeke) with a view to implementing the INSET strategy nationally. Modules for teachers were developed in three key areas: mathematics, English and pedagogy. To date 2,052 primary school teachers from 141 schools across the 7 Districts have been participating in the pilot programme. The MoEVT, together with the Prime Minister’s Office Regional Administration and Local Government (PMO-RALG) have now committed to scaling up this model nationally from January 2013. Although initial monitoring of the programme has taken place and early indications are that the pilot is progressing well, a more rigorous and detailed evaluation is needed to inform the Government’s scale-up plans ensuring the scale-up is efficient, effective and sustainable. Purpose The purpose of this consultancy is to provide technical support to the MoEVT and PMO-RALG to evaluate implementation of the school based INSET pilot programme in the seven Councils with a view to informing the national roll out of this programme. In particular the evaluation should:

Assess the effectiveness and efficiency of the pilot programme particularly with regards to improving teaching methodologies; enhancing knowledge of the subject matter by teachers and; making a difference to learning experiences, and where possible learning outcomes, of children;

Identify gaps within the current INSET model with a focus on: a) the INSET delivery system itself, including the support mechanisms for teachers and; b) the content of INSET programme materials and;

Provide lessons learnt and clear concrete conclusions on how to improve or tweak the existing model with a view to national roll-out.

50

The primary audience for the evaluation will be the Government of Tanzania and more specifically the MoEVT and PMO-RALG as these are the Ministries with responsibility for INSET. The seven participating Districts will also be interested to know how to improve support and scale-up of the INSET model. DPs and other important education stakeholders including parents and community leaders will be amongst other interested parties. This evaluation should be independent and should look at the entire INSET pilot programme from its design and implementation to its completion. Evaluation Questions Based on the purpose and scope of the evaluation, the following questions are expected to guide the evaluation. The questions are not exhaustive. Effectiveness

To what extent did the INSET pilot programme achieve its desired results?

Was the programme designed in such a way as to meet objectives of the INSET strategy?

Are the INSET teaching materials fit for purpose? Efficiency

Was the programme designed and implemented in a cost-efficient way?

Were the teachers able to study the materials in the allocated time?

Is the INSET system design an efficient one? Impact

Did the programme enhance teaching skills especially in maths and English?

Are teachers using more child centred approaches?

Did the learning experience of children improve?

Have communities and parents seen any changes in how their children are learning? Moving forward

What changes, if any, need to be made to the INSET delivery system?

What changes, if any, need to be made to the INSET materials?

What gaps, if any, exist which could threaten the sustainability of this programme? How can those gaps be addressed? Evaluation methodology The evaluation process will comprise of the following three broad phases.

Inception – includes creation and approval of a detailed evaluation work-plan;

Data collection and field visits – encompasses both document review and field visits. Classroom observations will be one important method used particularly with regards to analysing the change in classroom practices and;

51

Analysis and reporting – incorporates presentation of findings, delivery of a draft, capturing of feedback and production of final written report. Specific activities the evaluators will need to undertake include:

1. Working in close collaboration with the national INSET management team – co-ordinated and led by the MoEVT and including PMO-RALG and UNICEF– throughout the entire evaluation process;

2. Engaging other key Government partners including the Tanzania Institute of Education (TIE), Teachers’ Colleges, Dar es Salaam University College of Education (DUCE), Development Partners (DPs) and Civil Society Organisations in the evaluation;

3. Developing a detailed evaluation plan which will include methodologies to be used; 4. Reviewing existing literature and materials on the INSET pilot programme including

baseline studies, the teacher competency framework, teachers’ modules on maths, pedagogy and English, INSET guidelines, the INSET strategy (including its M&E framework), the Joint Field Monitoring Report, the Training Report and other relevant documents. The literature review will not be confined to Tanzania only, the evaluators should also look at studies and materials from other countries for comparisons and recommendations;

5. Identifying areas in: the design of the INSET model and; within the INSET learning materials that need modification, strengthening and/or emphasis so as to be addressed in the scaling up plan;

6. Training of a team of data collectors who will support the evaluators in gaining information and insight from the 7 pilot District Councils. This team will consist of:

7. education professionals who: are both experienced in the field of teacher education and; have been involved in the design, implementation and management of the pilot programme;

8. Providing recommendations on how best the programme can be scaled up with an emphasis on sustainability and;

9. Organising a participatory workshop with key education stakeholders to present key evaluation findings and recommendations for future scale-up.

52

Expected Deliverables and Timeframe

Deliverable Timeline

1. An inception report which should include

A detailed evaluation work-plan;

Literature review;

Evaluation question;

Identification of key informants;

Data collection methods and reasons for selection (with reference to the key informants) and;

A presentation of initial findings and provisional recommendations

22 September 2012

2. Draft report which includes:

An Executive Summary;

Details of the INSET pilot programme;

Scope of the evaluation;

Evaluation methodology and guiding principles;

Findings;

Lessons learned;

Recommendations which should include a scale-up plan and budget that considers, amongst other things, an incentive package for teachers and strengthening INSET co-ordination at all levels;

Methodological constraints and;

Additional background data such as e.g. questionnaires used

12 November 2012

3. Final Evaluation report

Encompassing all key sections required in the draft report and additional stakeholder feedback.

09 December 2012

Contract Management UNICEF’s Education Specialist will be responsible for the day-to-day management of this contract. The evaluators will present the deliverables to the UNICEF Education Specialist and the Director of the Teacher Education Department at the MoEVT together with the Government INSET management team. Payment of the contract will be made subject to approval by both the UNICEF Education Specialist and the Director of Teacher Education at the MoEVT. UNICEF and the MoEVT expect the evaluators to lead this work but will support the consultants by:

ensuring that all relevant contacts and information are available;

making available key documentation;

facilitating field visits;

providing office space for meetings;

facilitating the organisation of the consultation stakeholder feedback workshop and;

performing other technical and administrative duties as required.

53

UNICEF and MoEVT will not provide office space to the evaluators. Evaluation team composition and experience The team should be comprised of at least two evaluation professionals including one lead evaluator. S/he will responsible for managing the work of the team. The lead evaluator should be an international consultant. The second evaluator should be a national consultant. The lead evaluator will have the overall responsibility for the timely and quality submissions of all the evaluation deliverables. S/he may also participate in the selection of the national consultant. All reports, briefing and presentations will be in English and shall be properly edited and presented to allow for direct publication upon reception. The lead evaluator (international) must meet the following criteria to be eligible to apply for the assignment: 1. A minimum of a Master’s Degree in Education, Social Sciences or any related field; 2. A minimum of 10 years of professional experience in the field of evaluation especially in education and preferably within the area of teacher education and learner centred approaches; 3. Good understanding of education system, management and planning as well as classroom practices; 4. Excellent report writing skills; 5. Proficiency in English is essential, knowledge of Kiswahili is desired; 6. Demonstrated ability to deliver quality results within strict deadlines; 7. Experience in managing and undertaking complex evaluations and; 8. Previous working experience with Governments and within the UN system and familiarity with Tanzania’s education system and culture will be considered an asset. The national evaluator must meet the following criteria to be eligible to apply for the assignment: 1. A minimum of a Master’s Degree in Education, Social Sciences or any related field; 2. A minimum of 7 years of professional experience in the field of education and specifically teacher education training and material development for teachers; 3. Good understanding of classroom practices in the Tanzanian context and Africa in general; 4. Experience in undertaking evaluations; 5. Excellent report writing skills; 6. Proficiency in English and Kiswahili is essential; 7. Demonstrated ability to deliver quality results within strict deadlines and; 8. Previous working experience with Governments and within the UN system and familiarity with Tanzania’s education system and culture will be considered an asset. Candidates for both positions will be required to provide:

A CV and covering letter which should include information about daily fee rates;

A concept note underlining the approaches they would take to conduct the evaluation and;

Example evaluation studies they have already undertaken.

54

How to apply Candidates who have the required qualifications and experience, should submit their applications online to the Human Resources Manager, UNICEF Tanzania Country Office, Dar es Salaam indicating clearly Consultancy Vacancy Notice number 2012-001 Email: hr.tanzania@unicef.org by the 29 August 2012. All applications should be accompanied with a copy of the CV and a list of similar work that the consultant has previously done that is relevant to this assignment. (Please note that only short-listed candidates will be contacted). Female candidates are encouraged to apply. Closing date for applications: 29 August 2012. UNICEF IS A SMOKE-FREE ENVIRONMENT

55

APPENDIX 2: CLASSROOM OBSERVATION SCHEDULE (FREQUENCY)

Observer’s Name: _________________________________________________________ A. General Information

Name of school:

District:

Date:

B. Details of lesson

Teacher’s Name:

Subject:

Class:

Start Time: End Time:

No. of boys present:

No. of girls present:

No. of boys absent:

No. of girls absent:

No. of pupils with additional learning needs:

Lesson topic:

C. Use of Textbooks

Pupil-textbook ratio in classroom: ………. / ……….

Condition of textbooks (tick one box and comment if appropriate)

Good Satisfactory Poor

56

D. Lesson planning Using the following scale, please indicate the quality of the lesson plan: 1 = unsatisfactory, 2 = satisfactory, 3 = good, 4 = very good

1 2 3 4

Clarity of learning objectives

Lesson timeline

Range of teaching and learning activities

Use of instructional materials

Additional learning needs

Assessment of learning

Setting of homework (if appropriate)

E. Judging the quality of teaching and learning Please refer to the descriptors on pages 6 to 9 when applying judgements: 1 = behaviour never observed 2 = behaviour rarely observed (i.e. once or twice) 3 = behaviour occasionally observed (i.e. 4 or 5 times) 4 = behaviour consistently observed

Section 1: Demonstrating skills in lesson introduction and development

1 2 3 4

1. States objectives and provides overview of lesson

2. Checks for prior knowledge

3. Explains material accurately and clearly

4. Emphasises key points of the lesson

5. Uses a range of instructional materials

6. Makes effective use of chalk/white board

7. Creates positive classroom climate

8. Knows and uses pupil names

9. Uses paired or group work

10. Arranges classroom to facilitate learning

11. Uses plenary to summarise, consolidate and extends learning

57

Section 2: Demonstrating skills in questioning

1 2 3 4

12. Uses cued elicitation for repetition and drilling

13. Asks closed questions

14. Asks open-ended questions

15. Calls on pupils individually to answer questions

16. Asks pupils to demonstrate in front of class

17. Includes pupils with additional learning needs

Section 3: Demonstrating skills in feedback

1 2 3 4

18. Acknowledges pupil answers

19. Probes pupil answers

20. Comments on pupil answers

21. Builds pupil answers into subsequent questions

22. Encourages pupils to ask questions

23. Moves around room to interact with pupils to provide spoken and/or written feedback

Section 4: Demonstrating skills in classroom management

1 2 3 4

24. Relates well to learners

25. Effectively manages the class

26. Effectively manages timing of lesson

58

Observation Descriptors Demonstrating skills in lesson introduction and development Statement Descriptor

1. States objectives and provides overview of lesson

Learning objectives are incorporated into a lesson plan and clearly stated at the beginning and various stages of a lesson

2. Checks for prior knowledge Teacher asks pupils about previous

work covered in the topic and questions them about their understanding

3. Explains material accurately and clearly Teacher explanation is accurately and clearly presented with good signposting and makes strong connections to pupil experience Good examples, analogies, visual aids or other devices used to help the pupils understand

4. Emphasises key points of the lesson Teacher summarises what has been learnt at various stages throughout the lesson

5. Uses a range of instructional materials Teacher makes effective use of audio-visual aids and ICT where available Instructional aides, such as maps, tables, posters, pictures and charts, are clearly displayed so that all pupils can see and use them Teacher makes effective use of teacher’s guide/textbook

6. Makes effective use of chalk/white board Teacher’s writing and diagrams are clear and effectively laid out

7. Creates positive classroom climate Teacher conveys enthusiasm through voice and body language

59

Teacher uses encouragement and praise to give positive feedback Teacher does not shout, make, hurtful/ embarrassing/humiliating remarks or use corporal punishment

8. Knows and uses pupil names Teacher calls on pupils by name to make a contribution to the lesson

9. Uses paired or group work Activities requiring pupil-pupil Interaction are built into lesson Pupils are clear about the purpose and outcomes of the paired or group work Pupils are trained in how to work in groups (e.g. how to ask questions, listen and respond to each other)

10. Arranges classroom to facilitate learning Layout of the classroom is appropriate to the learning task/activities Learning needs of all pupils accommodated by classroom layout

11. Uses plenary to summarise, consolidate and extend learning

Teacher uses plenary session to draw the whole class together, during and at the end of the lesson, to summarise, consolidate and extend what has been covered and direct pupils to the next stage of learning

Demonstrating skills in questioning

Statement Descriptor

12. Uses cued elicitation for repetition and drilling

Teacher uses a mid-sentence rise in voice intonation to get a response from the pupils during an explanation or following a pupil answer The answer, usually in the form of a choral answer, takes the form of a repetition or completion of a phrase or word, initiated by the teacher

60

13. Asks closed questions Teacher asks test questions calling for a

single ‘yes/ no’ answer or offering facts

14. Asks open‑ended questions Teacher asks questions to which there is more than one answer Teacher asks questions which encourage speculation and require more than a ‘yes’ or ‘no’ answer or the recall of information

15. Calls on pupils individually to answer questions

Teacher encourages individual rather than choral responses to question Teacher ensures all pupils have a chance to respond to a question In mixed classes distribution of questions is roughly equal to the gender make-up of the class

16. Asks pupils to demonstrate in front of class

Teacher calls on pupils to answer questions, explain ideas and report back on activities in front of class

17. Includes pupils with additional learning needs

Teacher involves pupils with additional learning needs matching question to their abilities

Demonstrating skills in feedback

Statement Descriptor

18. Acknowledge pupil answers

Teacher indicates that their reply to question was appropriate with, for example, a ‘yes’, ‘no’, ‘ok’ response

19. Probes pupil answers Teacher stays with the same pupil and asks for further elaboration or explanation as to how they arrived at the answer

20. Comments on pupil answers Teacher exemplifies, expands, justifies or provides additional information on a pupil answer

61

21. Builds pupil answer into subsequent question(s)

Teacher asks a follow-up question which builds on pupil answer

22. Encourages pupils to ask questions Teacher encourages pupils to ask questions directed to both the teacher and other members of the class

23. Moves around to interact with pupils to provide spoken and/or written feedback

Teacher provides spoken comments on pupil work individually or in groups to inform learning Written feedback gets beyond the simple marking of work to provide detailed formative feedback

Demonstrating skills in class management and control Statement Descriptor

24. Relates well to pupils Teacher has good rapport with pupils

Teacher demonstrates enthusiasm, commitment and warmth

25. Effectively manage the class Teacher uses positive behaviour management strategies and deals with pupils with respect and dignity Teacher has clear ground rules for classroom behaviour Teacher stops potential discipline problem from escalating Teacher reinforces good behaviour with praise

26. Effectively manages timing of lesson Teacher sets clear, and restricted, goals Teacher effectively manages transitions between lesson activities High pupil levels of time on task

62

APPENDIX 3: TIMELINE OBSERVATION SCHEDULE

Observer: Date:

School: District:

Lesson start time: Lesson end time:

Subject Grade:

No. of boys present: No. of girls:

No. of boys absent: No. of absent:

Lesson topic:

For each FIVE MINUTE section of the lesson, please record on the checklist

the MAIN teaching and learning activities you observe as defined by the

descriptors on page 2.

Please use the note section to record activities NOT covered in the checklist

and to provide further contextual information on your observations.

Where code switching between languages takes place, please record this in

the notes section.

63

Timeline Descriptors

ACTIVITY Descriptor

1. Teacher explanation, question & answer

Lectures to class interspersed with question and answer routines

2. Teacher rote/chorus response

Uses cued elicitation for repetition and drilling of facts and information

3. Teacher working on chalk/white board

Writing or drawing on the chalk or white board

4. Teacher reading to whole class

Reading to whole class from textbook , chalk/ whiteboard or visual aid

5. Pupil reading to whole class

Pupil directed by teacher to read to the whole class from textbook , chalk/white board or visual aid

6. Pupils working individually

Pupils asked to work individually on task from textbook, worksheet, chalk/whiteboard

7. Pupils working in pairs/groups

Pupils asked to collaborate together on task or activity in pairs/groups

8. Pupil demonstrating to whole class

Pupils asked to demonstrate answer or make a presentation to whole class

9. Teacher reviews topic Lesson topic reviewed to consolidate learning

10. Teacher marking work

Teacher marks exercise books while class complete set task individually or in groups

11. Class management Teacher attention focused on managing pupil behaviour

12. Class administration

Administrative tasks e.g. taking register, giving out teaching materials, organising group work, setting up equipment

13. Interruption to lesson

Lesson stopped due to interruption e.g. visitor enters the room

14. Pupils off-task Pupils not engaged in teaching and learning activity e.g. lesson ends early

64

Time - 1 to 5 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

65

Time - 6 to 10 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

66

Time - 11 to 15 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

67

Time - 16 to 20 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

Key: A1 = Teacher explanation/question & answer; A2 = Teacher rote/chorus responses; A3 = Teachers writing on chalkboard; A4 = Teacher reading to whole class; A5 = Pupil reading to whole class; A6 = Pupils working individually; A7 = Pupils working in pairs/groups; A8 = Pupil demonstrating to class; A9 = Teacher reviews lesson topic; A10 = Teacher marking work; A11 = Class Management; A12 = Class administration; A13 = Interruption to lesson; A14 = Pupils off-task

68

Time - 21 to 25 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

69

Time - 26 to 30 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

70

Time - 31 to 35 minutes Notes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

71

Time - 36 to 40 minutes

MAIN ACTIVITY(IES)

1.Teacher explanation/ question & answer

2. Teacher rote/chorus responses

3. Teacher working on chalk/white board

4. Teacher reading to whole class

5. Pupil reading to whole class

6. Pupils working individually

7. Pupils working in pairs/groups

8. Pupil demonstrating to whole class

9. Teacher reviews lesson topic

10. Teacher marking work

11. Class management

12. Class administration

13. Interruption to lesson

14. Pupils off-task

72

APPENDIX 4: COMPUTERISED OBERVATION SCHEDULE

Stage of the lesson:

Behaviour: Definition:

Whole class The teacher is addressing the whole class

Group work The pupils are working in groups

Individual The pupils are working on their own

Interruption An interruption to the lesson (e.g. another member of staff may come in)

Main behaviours:

Behaviour: Definition:

Explanation To pass on facts, opinions and ideas about a subject. Delivering new info.

Direction Designed to get pupils to do, not to say something (an order)

Question A question designed to elicit an answer

Open question Question calls for more than one answer

Cued elicitation Where teacher says a sentence and then repeats it and the pupils finish the sentence. E.g. The capital of Tanzania is. The capital of Tanzania is …’

Checks A quick knowledge check – usually ‘do you understand?’ or ‘yes?’

Seat work Pupils working individually on a task or exercise

Evaluates Teachers evaluates a pupil

Reads Teacher or pupil reading out loud.

Writes Teacher or pupil writing for rest of class to see.

Refocus To get the attention of a pupil back on the curriculum content (e.g. ‘quiet!’)

General talk A 'dump' category for any teacher talk not covered by the above

Answer A response to a question

Pupil initiation Pupil asks a question

Modifiers:

Evaluation type

Praise Praises in words or by expressing verbal affirmation

Accept Simply affirms that the student's response is correct or not

Comments Teacher rephrases or elaborates on a pupil answer

Probe Teacher requiring further information from a pupil’s answer by staying with same pupil or asking others in class (e.g. ‘Why? How?’)

73

APPENDIX 5: HEAD TEACHER INTERVIEW School-Based INSET Programme Programu ya Mafunzo ya Walimu Kazini Elimu ya Msingi (MWAKEM)- Ngazi ya Shule (Tanzania). Head Teacher Interview

Name of Interviewer:

Date:

District/Municipal Council: School:

Ward Start Time:

Gender of head teacher: Finish Time:

Years as head teacher:

INTRODUCTION STATEMENT (UTANGULIZI)

This interview aims to find out how the new MOEVT School-based In-service Education and Training (INSET) programme in English, Mathematics and Pedagogy can be improved (Mahojiano haya yanalengo la kupata taarifa kuhusu mafunzo ya walimu kazini ngazi ya shule (MWAKEM) ili kuboresha ufundishaji wa English, Hisabati na Njia za kufundishia na kujifunza. Tathmini hii inafanywa na kusimamiwa na Wizara ya Elimu na Mafunzo ya Ufundi kwa kushirikiana na Ofisi ya Waziri Mkuu-TAMISEMI). The survey is being undertaken by the Ministry of Education and Vocational Training (MOEVT) and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG). In order to do this we require accurate answers to the following questions. (Ili kufanikisha utafiti huu, tunategemea majibu sahihi kwa maswali yafuatayo). The MOEVT and PMO-RALG thanks you in advance for the information provided (Kwa msingi wa malengo hayo Wizara ya Elimu na Mafunzo ya Ufundi na TAMISEMI wanatanguliza shukurani kwa taarifa utakayotoa). The information you provide is ONLY for research purposes (Taarifa utakayotoa itatumika kwa malengo ya kufanikisha tathmini hii tu na si kwa malengo mengine). YOUR NAME WILL NOT BE RECORDED IN THIS INTERVIEW (Jina lako halitaonyeshwa popote). If the respondent is unwilling to answer questions, please give details for non-response:

74

1. What school-based INSET has your school been involved with this last year? Shule yako imeshiriki mafunzo ya aina gani ngazi ya shule katika kipindi cha mwaka mmoja uliopita?

In-Service Training Received Mafunzo kazini yaliyotolewa

Yes

No

Length of Training (months)

Materials given during training (Mf. Moduli, miongozo, vipeperushi n.k)

MOEVT INSET in English Mathematics and Pedagogy

Teacher Resource Centre course Mafunzo katika vituo vya walimu (TRCs)

University courses Mafunzo ya Vyuo Vikuu

Non-Government Organisation courses Mafunzo yanayotolewa na asasi zisizo za Kiserikali

Private provider courses Mafunzo yanatolewa na watu/taasisi binafsi

Other (specify) Mafunzo mengineyo (fafanua)

2. In your opinion, which of the INSET activities engaged in by your staff have had the greatest

benefit to the pupil through increasing teacher capability? (Kwa maoni yako, ni maeneo gani

katika mafunzo ya walimu kazini yamesaidia wanafunzi kujifunza vizuri?).

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

3. Can you give some examples of how the INSET programme has improved teacher

effectiveness? (Toa mifano inayothibitisha kwamba kuimarika kwa ufundishaji wa walimu hapa

shuleni ni kutokana na mafunzo ya walimu kazini).

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

75

4. Have your teachers who have attended the MOEVT INSET programme been observed? (Je walimu walioshiriki mafunzo ya MWAKEM wamekuwa wakiangaliwa wakati wanafundisha darasani?).

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ YES/NO (Please circle) If yes, please describe who observed your teachers and how effective you found the visits. (Kama jibu ni ndiyo eleza ni nani aliyekuwa anawaangalia na kuna manufaa gani kwa kufanya hivyo). ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

5. How useful has the content of the INSET programme been for meeting the training needs of your staff (e.g. modules in pedagogy, English and mathematics)? Kwa namna gani maudhui ya MWAKEM yamekidhi mahitaji ya walimu (mf. Moduli ya njia za ufundishaji, English na Hisabati

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

6. How effective has the delivery method of the INSET programme been for meeting the training needs of your staff (e.g. modules, cluster meetings, observations)? (Ni kwa kiwango gani njia za utoaji mafunzo ya walimu kazini zimekidhi mahitaji ya walimu (mf. Moduli, mikutano ya Klasta, Ufuatiliaji)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

76

7. When do teachers in your school undertake most of their INSET activities (e.g. during school

hours, after school, weekends, in school holidays)? (Ni wakati gani walimu wanajishughulisha na

mafunzo ya walimu kazini (mf. Wakati wa vipindi vya shule, baada ya vipindi, mwisho wa wiki

au wakati wa likizo?).

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

8. Have your INSET trained teachers delivered school-based training? (Je walimu waliopitia MWAKEM wanaendesha mafunzo ngazi ya shule?)

YES/NO (Please circle)

If yes, please describe what they did and how effective you found the training (Ikiwa ndiyo, tafadhali eleza ni mafunzo gani waliyotoa, na unaonaje mafanikio ya mafunzo hayo).

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 9. What are the impediments to providing your teaching staff with the best possible INSET?

[Please rank in order - 1 for the greatest impediment, 6 for the least] (Je kuna vikwazo gani

katika kufanikisha utoaji wa mafunzo bora kwa walimu kazini (Toa mifano ya vikwazo. Ipange

kwa uzito).

Travel time to INSET cluster – Muda wa kwenda katika Klasta Cost of travel to courses – Gharama za kusafiri kufika kwenye Klasta Lack of INSET budget – kukosekana kwa bajeti ya mafunzo Lack of resources - kukosekana kwa vitendea kazi Lack of suitable relieving teachers – kukosekana kwa walimu wa kupokea vipindi Teacher workload – kupokea vipindi vingi kwa walimu INSET trained teachers leaving the school – kuhama kwa wawezeshaji Unwillingness of staff to participate - ushiriki mdogo wa baadhi ya walimu Other (specify) ______________ sababu nyinginezo (taja)

10. What training have you received for providing the MOEVT INSET in your school? (Je umepata

mafunzo yoyote ili kukuwezesha kuendesha mafunzo ya MWAKEM shuleni kwako?)

________________________________________________________________________________

________________________________________________________________________________

77

________________________________________________________________________________

__________________________________________________________________________

11. Do you have a Whole School Development Plan? (Je una mpango wa jumla wa maendelea

ya shule?)

YES/NO (Please circle)

If yes, does the MOEVT INSET feature in it and what priorities have been set? (Ikiwa ni ndiyo je MWAKEM imo katika mpango? Imewekewa vipaumbele gani?

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

12. Are there any other comments you wish to make about the MOEVT INSET in English

mathematics and Pedagogy programme? Je una maoni zaidi kuhusu MWAKEM katika masomo

ya (English, Hisabati na Njia za Ufundishaji)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering the interview questions

78

APPENDIX 6: TEACHER INTERVIEW

Interview for INSET trained teachers School-Based INSET Programme

Name of Interviewer:

Date:

District:

School:

Ward:

Start Time:

Gender of teacher:

Finish Time:

Years in teaching:

INTRODUCTION STATEMENT (UTANGULIZI)

This interview aims to find out how the new MOEVT School-based In-service Education and Training (INSET) programme in English, Mathematics and Pedagogy can be improved (Mahojiano haya yanalengo la kupata taarifa kuhusu mafunzo ya walimu kazini ngazi ya shule (MWAKEM) ili kuboresha ufundishaji wa English, Hisabati na Njia za kufundishia na kujifunza. Tathmini hii inafanywa na kusimamiwa na Wizara ya Elimu na Mafunzo ya Ufundi kwa kushirikiana na Ofisi ya Waziri Mkuu-TAMISEMI). The survey is being undertaken by the Ministry of Education and Vocational Training (MOEVT) and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG). In order to do this we require accurate answers to the following questions. (Ili kufanikisha utafiti huu, tunategemea majibu sahihi kwa maswali yafuatayo). The MOEVT and PMO-RALG thanks you in advance for the information provided (Kwa msingi wa malengo hayo Wizara ya Elimu na Mafunzo ya Ufundi na TAMISEMI wanatanguliza shukurani kwa taarifa utakayotoa). The information you provide is ONLY for research purposes (Taarifa utakayotoa itatumika kwa malengo ya kufanikisha tathmini hii tu na si kwa malengo mengine). YOUR NAME WILL NOT BE RECORDED IN THIS INTERVIEW (Jina lako halitaonyeshwa popote). If the respondent is unwilling to answer questions, please give details for non-response:

__________________________________________________________________________________________________________________________________________________

79

1. How would you rate your current level of motivation in your teaching post? (Please √ response) (Unapimaje kiwango chako cha motisha katika kazi ya Ualimu?) (weka tiki(√)kwenye jibu ulilochagua)

1 Very motivated

(Kiwango cha juu)

2 On the whole motivated

(kawaida)

3 Not at all motivated

(hakuna motisha)

2. Have you attended any of the following professional develop programmes in the last 2 years? (Umewahi kuhudhuria mafunzo yoyote ya kujiendeleza katika kipindi cha miaka miwili iliyopita?)

In-Service Training Received Mafunzo kazini yaliyotolewa

Yes

No

Length of Training (months)

Materials given during training (Mf. Moduli, miongozo, vipeperushi n.k)

MOEVT INSET in English Mathematics and Pedagogy (MWAKEM)

Teacher Resource Centre course Mafunzo katika vituo vya walimu (TRCs)

University courses Mafunzo ya Vyuo Vikuu

Non-Government Organisation courses Mafunzo yanayotolewa na asasi zisizo za Kiserikali

Private provider courses Mafunzo yanatolewa na watu/taasisi binafsi

Other (specify) Mafunzo mengineyo (fafanua)

3. In your opinion, which of the training activities provided in the MOEVT INSET programme (e.g.

English, mathematics, pedagogy modules, cluster meetings, observations) have been the most

effective? (Kwa maoni yako ni maeneo gani yaliyomo katika mpango wa MWAKEM (mf.moduli

za English,Hisabati na Ujuzi wa kufundisha kwa umahiri;Mikutano ya Klasta;kumtazama

Mwalimu akifundisha) yamekuwa ya manufaa? )

80

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

4. Can you give some examples of how the MOEVT INSET programme has improved your

teaching? (Toa mifano ya jinsi gani MWAKEM imekusaidia kuboresha ufundishaji wako?)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

5. Can you give some examples of how the MOEVT INSET programme has improved pupil learning? (Toa mifano ya jinsi gani MWAKEM imeboresha ujifunzaji wa wanafunzi wako)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

6. Have you been involved in delivering the MOEVT INSET in English, mathematics and pedagogy

to other teachers in your school? (Je, umeshiriki katika utoaji wa mafunzo ya MWAKEM katika

masomo ya English,Hisabati na Ujuzi wa kufundisha kwa umahiri kwa walimu wengine shuleni

kwako?)

YES/NO (Please circle)

If yes, please describe what you did, who was there and how well the training went. (kama ndiyo

eleza ulifanya nini, nani walihudhuria na mafunzo yalikuwaje?)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

7. Have you been observed teaching in your school as part of the MOEVT INSET? (Umewahi kuan

galiwa ukifundisha darasani kama sehemu ya mafunzo ya MWAKEM?)

YES/NO (Please circle)

If yes, by who and how helpful did you find the observation? (Kama ndiyo uliangaliwa na nani?

Na zoezi hilo lilikuwa na manufaa kiasi gani?

81

___________________________________________________________________________________________________________________________________________________________________________________________________________________________

8. What aspects of the MOEVT INSET have you found the most useful? (e.g. training modules, observations, mentoring, support groups, tutorial groups, assignments). (Vipengele gani vya MWAKEM ulivyoviona kuwa na manufaa zaidi mf. Moduli, kumwangalia mwalimu akifundisha, Walimu mahiri, majadiliano katika vikundi, ujifunzaji katika vikundi, kazi za mazoezi)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

9. When do you undertake most of your INSET? (e.g. during school hours, after school, weekends,

in school holidays)? (Ni wakati gani uliotumia zaidi katika kujifunza MWAKEM (mf. wakati wa

saa za masomo, baada ya saa za masomo, mwishoni mwa juma, wakati wa likizo)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

10. Are there any other comments you wish to make about the MOEVT INSET in English,

mathematics and pedagogy you have received? (Toa maoni mengine kuhusu MWAKEM Katika

masomo ya English,Hisabati na Ujuzi wa kufundisha kwa umahiri)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering the interview questions

82

APPENDIX 7: PUPIL FOCUS GROUP INTERVIEW Focus group interview with Grade 6 students School-Based INSET Programme Questionnaire

Note to interviewer: please choose 6 students (3 boys/3 girls) and interview them in a quiet area of the school with no teachers present

Name of Interviewer:

Date:

School:

Start Time:

District:

Finish Time:

INTRODUCTION STATEMENT This survey aims to find out what you think about your lessons, what you like about them and how they can be made better. The information will be used by the Ministry of Education and Vocational Training to help improve your school. (Madhumuni ya tathmini hii ni kutaka kufahamu unayafikiria vipi masomo yako na kwa namna gani yanaweza yakaboreshwa. Taarifa tutakazozipata zitaisadia Wizara ya Elimu na Mafunzo kuiboresha shule yako). Anything that you tell us will not be reported back to your teachers and we would like you to be honest in your answers. (Jambo lolote utakalo tuambia ni taarifa kati yetu na wewe, walimu hawatafahamishwa. Kwa hiyo, tafadhali tueleze ukweli wakati wa kujibu maswali haya). YOUR NAMES WILL NOT BE RECORED IN THIS INTERVIEW (Jina lako alitaandikwa popote katika mahojiano haya). If respondents are unwilling to answer questions, please give details for non-response:

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

83

Classroom climate

A1. If I came to your school, what would I like? What would I not like? (Mtu akija shuleni kwako,

ni kitu gani kitamvutia? Na kitu gani hakitamvutia?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ A2. What subjects do you like the most? Please tell me why you like them. (Ni masomo gani

unayapenda zaidi? Tafadhali niambie ni kwanini unayapenda?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

A3. What kinds of things do you like doing in mathematic lessons? (Ni mambo gani unapenda

kuyafanya katika somo la Hisabati?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ A4. What kinds of things do you like doing in English lessons? (Ni mambo gani unapenda

kuyafanya katika somo la English?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

A5. Have you seen any changes in the way your teacher has been teaching since last year? If so,

what are they doing differently? (Je, unaona mabadiliko yoyote ya namna mwalimu wako

anavyofundisha tangu mwaka jana? Kama ndiyo, je, ni mabadiliko gani?)

___________________________________________________________________________________________________________________________________________________________________________________________________________________________

84

B. Teaching & learning (Ufundishaji na ujifunzaji)

B1. Can you tell me about the last time you worked in a group*? In what subject(s) did you do

the group work? What were you asked to do? How many pupils were in the group? Did you

work well together? (Unaweza kunieleza kuhusu ushiriki wako katika vikundi vya majadiliano?

Ni masomo yapi mlijadiliana? Mlikuwa wangapi katika kikundi? Mlishirikiana vizuri?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

B2. What one thing would help you become a better learner? (Je, unafikiri ni jambo gani

muhimu linaloweza kukusaidia kujifunza vizuri?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

B3. Are there enough books to help you learn English and Mathematics in class? (Je, kuna vitabu

vya kutosha kukusaidia kujifunza English na Hisabati darasani?)

(a) Yes (b) No (Circle)

If No, how do you learn English and Mathematics (Kama jibu ni hapana unajifunzaje masomo hayo?) _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

85

B4. Have your lessons been more interesting since last year? If so, in what ways? (Je, masomo

yanavutia tangu mwaka uliopita? Kama ndiyo, ni kwa namna gani?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ *Please explain what you mean by group work to the students i.e. working collaboratively with

other students in a pair or group in class (*tafadhali eleza (fafanua) maana ya kazi za makundi,

kwa mfano kufanya kazi pamoja katika jozi au kundi darasani)

C. Classroom interaction (Mawasiliano na mahusiano darasani)

C1. When the teacher asks a question does she/he usually ask an individual pupil or the whole

class to answer? (Mwalimu aulizapo swali darasani, je huuliza kwa mwanafunzi mmoja mmoja

au kwa darasa zima?) (Angalizo: Zingatia kama wanaitwa kwa majina na kwa kuzingatia jinsia)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C2. Do you like being asked to answer a question in class? If not, why not? (Je, unapenda

kuulizwa swali darasani? Kama ni hapana, kwanini hupendi?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C3. What normally happens if a pupil gets an answer right? (Ni jambo gani kwa kawaida

hutokea kama mwanafunzi atajibu swali kwa usahihi?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

86

C4. What normally happens if a pupil gets an answer wrong? (Ni jambo gani kwa kawaida

hutokea kama mwanafunzi hatajibu swali kwa usahihi?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C5. What happens if a pupil misbehaves in class? (Nini kinafanyika kama mwanafunzi

ataonesha utovu wa nidhamu darasani?)

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering my questions.

87

APPENDIX 8: SCHOOL COMMITTEES/PARENTS Interview for School Committees/Parent Association School-Based INSET Programme

Name of Interviewer:

Date:

District:

School:

Ward:

Start Time:

Gender of teacher:

Finish Time:

Years in teaching:

INTRODUCTION STATEMENT (UTANGULIZI)

This interview aims to find out how the new MOEVT/PMO-RALG School-based In-service Education and Training (INSET) programme in English, Mathematics and Pedagogy can be improved (Mahojiano haya yanalengo la kupata taarifa kuhusu mafunzo ya walimu kazini ngazi ya shule (MWAKEM) ili kuboresha ufundishaji wa English, Hisabati na Njia za kufundishia na kujifunza. Tathmini hii inafanywa na kusimamiwa na Wizara ya Elimu na Mafunzo ya Ufundi kwa kushirikiana na Ofisi ya Waziri Mkuu-TAMISEMI). The survey is being undertaken by the Ministry of Education and Vocational Training (MOEVT) and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG). In order to do this we require accurate answers to the following questions. (Ili kufanikisha utafiti huu, tunategemea majibu sahihi kwa maswali yafuatayo). The MOEVT and PMO-RALG thanks you in advance for the information provided (Kwa msingi wa malengo hayo Wizara ya Elimu na Mafunzo ya Ufundi na TAMISEMI wanatanguliza shukurani kwa taarifa utakayotoa). The information you provide is ONLY for research purposes (Taarifa utakayotoa itatumika kwa malengo ya kufanikisha tathmini hii tu na si kwa malengo mengine). YOUR NAME WILL NOT BE RECORDED IN THIS INTERVIEW (Jina lako halitaonyeshwa popote). If the respondent is unwilling to answer questions, please give details for non-response:

88

1. Are you aware of the MOEVT’s in-service training for teachers that has been taking place in your school since last year? (Je unafahamu kuhusu mafunzo ya walimu kazini “MWAKEM” ambayo yanaendelea hapa shuleni tangu mwaka jana?)

YES/NO (Please circle)

If yes, what do you know about the training that has been provided? (Kama ndiyo, unafahamu

nini kuhusu mafunzo hayo?)

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

__________________________________________________________________________

2. Have you attended any meetings or been involved in any discussions about the MOEVT Pilot INSET in your school? (Je, umewahi kushiriki kikao au kuhusishwa katika majadiliano yanayohusu mafunzo ya walimu kazini hapa shuleni?)

YES/NO (Please circle)

If yes, what was the meeting and what was discussed? (Kama ndiyo, kikao hicho kilijadili nini?)

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

__________________________________________________________________________

3. Have you been involved in discussions of the whole school development plan? (Je umekuwa

ukishiriki katika kujadili mpango wa jumla wa maendeleo ya shule?)

YES/NO (Please circle)

If yes, how were you involved and has staff INSET been discussed in developing the plan? (Kama

ndiyo, ulishiriki vipi? Je mafunzo ya walimu kazini yalijadiliwa na kuingizwa kwenye mpango

wa shule?)

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

__________________________________________________________________________

89

4. Have you noticed any changes in the way teachers are teaching in your school? (Je kuna

mabadiliko unayoyaona katika kufundisha na kujifunza hapa shuleni?)

YES/NO (Please circle)

If yes, what changes have you noted? (Kama ndiyo, ni mabadiliko gani unayaona?)

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

__________________________________________________________________________

5. Do you think there has been any improvement in the quality of education your children are receiving since the last year? (Je unadhani kumekuwepo na ongezeko la ubora wa elimu wapatayo watoto shuleni tangu mwaka jana?)

YES/NO (Please circle)

If yes, what kinds of improvements have you noticed? (Kama ndiyo, ni vitu gani vimeboreshwa?)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

6. Are there any other comments you wish to make about your school’s involvement in the

MOEVT INSET? (Toa maoni mengineyo kuhusu ushiriki wa shule yako katika MWAKEM)

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering the interview questions

90

APPENDIX 9: DISTRICT OFFICER AND EDUCATION OFFICERS

School Based INSET Programme Interview for District Education Officers/School Inspectors /District INSET

Coordinators /Ward Education Coordinators

Name of Interviewer:

Date:

District: Start Time:

Ward:

Finish Time:

Male/female (Please circle):

Years as District Officer/Ward Coordinator/Inspector

INTRODUCTION STATEMENT (UTANGULIZI)

This interview aims to find out how the new MOEVT School-based In-service Education and Training (INSET) programme in English, Mathematics and Pedagogy can be improved (Mahojiano haya yanalengo la kupata taarifa kuhusu mafunzo ya walimu kazini ngazi ya shule (MWAKEM) ili kuboresha ufundishaji wa English, Hisabati na Njia za kufundishia na kujifunza. Tathmini hii inafanywa na kusimamiwa na Wizara ya Elimu na Mafunzo ya Ufundi kwa kushirikiana na Ofisi ya Waziri Mkuu-TAMISEMI). The survey is being undertaken by the Ministry of Education and Vocational Training (MOEVT) and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG). In order to do this we require accurate answers to the following questions. (Ili kufanikisha utafiti huu, tunategemea majibu sahihi kwa maswali yafuatayo). The MOEVT and PMO-RALG thanks you in advance for the information provided (Kwa msingi wa malengo hayo Wizara ya Elimu na Mafunzo ya Ufundi na TAMISEMI wanatanguliza shukurani kwa taarifa utakayotoa). The information you provide is ONLY for research purposes (Taarifa utakayotoa itatumika kwa malengo ya kufanikisha tathmini hii tu na si kwa malengo mengine). YOUR NAME WILL NOT BE RECORDED IN THIS INTERVIEW (Jina lako halitaonyeshwa popote). If the respondent is unwilling to answer questions, please give details for non-response:

91

1. What do you think are the main training needs of primary teachers in your education district/ward? (Unafikiri ni mafunzo yapi yanayohitajika kwa walimu wa elimu ya msingi, katika Wilaya/Kata yako?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 2. In your opinion, which of the training activities undertaken by teachers in your ward/district (e.g. modules, cluster meetings, observations) as part of the INSET pilot have had a positive contribution on teaching and learning?[Kwa maoni yako unadhani shughuli gani za MWAKEM (k.m. za usomaji wa moduli, mikutano ya walimu katika Klasta, kuangalia walimu wakifundisha darasani) zimekuwa na mchango katika ufundishaji na ujifunzaji]. ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 3. How useful has the content of the pilot INSET modules been for meeting the training needs of teachers? (Walimu wanufaikaje na maudhui ya Moduli za MWAKEM?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 4. How useful has the delivery of the pilot INSET programme (e.g. distance learning materials, observations, cluster meetings) been for meeting the training needs of teachers? [Uwezeshaji wa jaribio la MWAKEM (kwa njia ya masafa, Klasta, kuwaangalia walimu wakifundisha darasani) umekuwa na faida gani katika kukabili mahitaji ya walimu ya kujifunza?] ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 5. When do teachers in your district/ward undertake most of their in-service training (e.g. during school hours, after school, weekends, in school holidays)?[Ni wakati gani walimu wanapata muda wa kujifunza katika Wilaya/Kata yako (wakati wa saa za kazi, baada ya saa za kazi, mwisho wa wiki na wakati wa likizo?)] ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

92

6. What role have you played in the pilot of the INSET programme in your Ward/District? (Umeshiriki namna gani katika mpango wa majaribio wa MWAKEM ndani ya Wilaya/Kata yako?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 7. What budget has been made available to you to deliver school-based INSET in your district/ward? (Ni kiasi gani katika bajeti kilichopangwa na kupatikana ili kufanikisha uendeshaji wa mafunzo ya MWAKEM katika Wilaya/Kata yako). ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

8. Are there any other comments you wish to make about School Based INSET for primary school teachers in your Ward/District? (Una maoni/ushauri wowote kuhusu MWAKEM katika kata/wilaya yako). ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering the interview questions

93

APPENDIX 10: TEACHER COLLEGE TUTORS

Interview for Teacher College Tutors School Based INSET Programme

Name of Interviewer:

Date:

Teacher College: Start Time:

District:

Finish Time:

Male/female (Please circle):

Years as College Tutor:

INTRODUCTION STATEMENT (UTANGULIZI)

This interview aims to find out how the new MOEVT/PMO-RALG School-based In-service Education and Training (INSET) programme in English, Mathematics and Pedagogy can be improved (Mahojiano haya yanalengo la kupata taarifa kuhusu mafunzo ya walimu kazini ngazi ya shule (MWAKEM) ili kuboresha ufundishaji wa English, Hisabati na Njia za kufundishia na kujifunza. Tathmini hii inafanywa na kusimamiwa na Wizara ya Elimu na Mafunzo ya Ufundi kwa kushirikiana na Ofisi ya Waziri Mkuu-TAMISEMI). The survey is being undertaken by the Ministry of Education and Vocational Training (MOEVT) and Prime Ministers’ Office–Regional Administration and Local Government (PMO-RALG). In order to do this we require accurate answers to the following questions. (Ili kufanikisha utafiti huu, tunategemea majibu sahihi kwa maswali yafuatayo). The MOEVT and PMO-RALG thanks you in advance for the information provided (Kwa msingi wa malengo hayo Wizara ya Elimu na Mafunzo ya Ufundi na TAMISEMI wanatanguliza shukurani kwa taarifa utakayotoa). The information you provide is ONLY for research purposes (Taarifa utakayotoa itatumika kwa malengo ya kufanikisha tathmini hii tu na si kwa malengo mengine). YOUR NAME WILL NOT BE RECORDED IN THIS INTERVIEW (Jina lako halitaonyeshwa popote). If the respondent is unwilling to answer questions, please give details for non-response:

94

1. What role have you played in the pilot MOEVT INSET programme? (Je, ni jukumu gani umelitekeleza katika mpango wa majaribio ya Mafunzo ya Walimu Kazini Elimu ya Msingi ‘MWAKEM’ ngazi ya Shule?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 2. In your opinion, which of the training activities undertaken by teachers in your district as part of the INSET pilot have had the greatest impact on teaching and learning? (Kwa maoni yako, je kazi gani za kimafunzo zilizofanywa na walimu katika wilaya yako zimekuwa na mchango mkubwa katika ufundishaji na ujifunzaji?) _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 3. How useful has the content of the pilot INSET modules been for meeting the training needs of teachers? (Je, maudhui ya moduli za mafunzo ya MWAKEM yamekidhi mahitaji ya mafunzo kwa walimu kazini?) _____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 4. How useful has the delivery of the pilot INSET programme (e.g. distance learning materials, observations, cluster meetings) been for meeting the training needs of teachers? [Je, mfumo uliotumika kutekeleza mpango wa majaribio wa MWAKEM (kwa mfano vifaa vya elimu masafa, ufuatiliaji na klasta ) umekidhi mahitaji ya mafunzo ya walimu kazini?] ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 5. When do teachers in your district undertake most of their in-service training (e.g. during school hours, after school, weekends, in school holidays)? (ni wakati gani walimu wa wilaya yako huhudhuria mafunzo ya ualimu kazini (kama vile masaa ya kazi, baada muda wa shule, siku za mwisho wa wiki, nyakati za likizo n.k?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

95

6. What training and support have you received to deliver the pilot MOEVT School-based INSET? (Je, umepata mafunzo na msaada gani katika kutekeza mpango wa majaribio wa MWAKEM ngazi ya shule?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 7. What challenges have you faced in delivering the pilot School-based INSET programme to teachers in your district? (Je, umekutana na changamoto gani katika utekelezaji wa mpango wa majaribio wa MWAKEM ngazi ya shule?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 8. Are there any other comments you wish to make about how the School-based INSET programme could be improved? (Je, una maoni mengine kuhusu namna MWAKEM inavyoweza kuboreshwa?) ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for answering the interview questions

96

APPENDIX 11: EVALUATION OF SCHOOL BASED INSET PILOT PROGRAMME Programme for data collector training workshop: 4 – 5 October 2012 The main purpose of the 3-day workshop will be to train data collectors in the skills required to gather and analyse classroom data and conduct interviews for the evaluation of the School Based INSET pilot programme. A central focus of the evaluation will be on what can be observed in the act of teaching and learning. Day 1 / Wednesday 3 October 2012 09.30 – 9.30: Registering Opening Remarks Background to study/design of research instruments 10.30 – 11.00: Break 11.00 – 12.30: Using the observation schedule 1 in the classroom 12.30 – 13.30 Lunch 13.30 – 16.00 Using observation schedule 2 in the classroom Establishing inter-rater reliability Day 2 / Thursday 4 October 2012 09.00: Piloting of instruments in schools 13.00 – 13.30 Lunch 13.30 – 16.00 Reflections on the school visit Day 3 / Friday 5 October 2012 09.00 – 10.30: Review of school-based observations 10.30 – 11.00: Break 11.00 – 12.30: Checking for inter-relater reliability 12.30 – 13.30 Lunch 13.30 – 16.00 Conducting of interviews in schools

Selecting sample of schools Protocols for school visits Visit guidelines, checklist and materials

97

Appendix 12: Cost Estimates for INSET training of one primary school teacher through School Based Approach

Unit cost of training one teacher going through school-based INSET per annum: 341,287?

a) Cost incurred by MoEVT: 319,787/

Material review, printing and distribution (modules, guidelines, NTCF, strategy) to district level-37,298/

Training of national and district facilitators-6,500/

Sensitization and awareness creation-1,000/

Professional support by district facilitators (tutors)-27,489/*

National and district level management and coordination (capacity building, facilitation and consultation meetings)-7,000/;

Ward and school level management and coordination-45,000/

Training of school-based facilitators (expert teachers/mentors)-195,000/

Monitoring and evaluation-6,500/*

b) Cost incurred by PMO-RALG at district, cluster and school levels: 21,500/

Distribution of materials to teachers (schools)-1,500/

Awareness and sensitization seminars to teachers-20,000/ Discussed and compiled by:

Ms. Maniza Ntekim (UNICEF);

Ms. Sarahflorentina Kironde (MoEVT);

Mr. Zebedayo Kyomo (DFID);

Mr. Fredrick Shuma (MoEVT);

Mr. Audax Tibuhinda (UNICEF) 15/11/2012 Note: 1. The calculations are based on the current number of 174,834 primary

school teachers; 2. *Continuing activities.

98

Indicative Direct Costs of Training a Student Teacher by Category and Specialization (In

Tshs): One Academic Year

S/N Item Grade III A Diploma Arts Diploma

Science

Special

Education

1. Salary 312,000.00 312,000.00 312,000.00 312,000.00

2. Electricity 34,476.00 34,476.00 34,476.00 34,476.00

3. Water 26,000.00 26,000.00 26,000.00 26,000.00

4. Meals 1,350,000.00 1,350,000.00 1,350,000.00 1,350,000.00

5. Books 214,000.00 234,000.00 284,000.00 244,000.00

6. Stationery 100,000.00 100,000.00 100,000.00 100,000.00

7. Special requiremets21 100,000.00 100,000.00 210,000.00 673,286.00

8. Accommodation 291,000.00 291,000.00 291,000.00 291,000.00

9. Block Teaching

Practice

304,000.00 304,000.00 304,000.00 304,000.00

10. First Aid 40,000.00 40,000.00 40,000.00 40,000.00

11. Uniforms 200,000.00 200,000.00 200,000.00 200,000.00

12. Living costs/stipend 400,000.00 400,000.00 400,000.00 400,000.00

13.

Caution money 40,000.00 40,000.00 40,000.00 40,000.00

Sub-total 3,411,476.00 3,431,476.00 3,591,476.00 4,014,762.00

Miscellenous 10% 341147.00 343147.00 359147.00 401476.00

Grand total 3,752,623.00 3,774,623.00 3,950,623.00 4,416,238.00

21

e.g laboratory facilities etc