1 examination impact: monitoring how teachers cope with change dianne wall tania hork lancaster...
Post on 18-Jan-2018
216 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
Examination Impact: Monitoring how teachers
cope with change
Dianne WallTania Horák
Lancaster University UK
2
Background
• Next Generation TOEFL (NGT) to be launched September 2005 (in certain sites only)
• Two impact studies commissioned by Educational Testing Service (producer of TOEFL)
3
Our impact study
• Where?Central and Eastern Europe
• Who?TOEFL preparation course providers
4
Phase 1 Phase 2 Phase 3
Baseline Transition Impact
Jan 03 to Oct 04 Oct 04 to Mar 06 After introduction of NGT
-Description of current TOEFL preparation classes-Class observations, interviews with teachers, DoS and students.-10 sites in 6 countries
-Tracking study -’Interviews’ via Internet
-Description of NGT preparation classes -Class observations, interviews with teachers, DoS and students.- same sites
5
Theoretical underpinnings 1) Henrichsen (1989) – Hybrid Model
of the Diffusion / Implementation Process
2) Messick (1996) - ‘evidential link’
6
Henrichsen’s model
Antecedents Process Consequences
7
Process • The Receivers• The Innovation• Characteristics of Communication• The Resource System • The User System • Inter-elemental factors
8
How our study fits the model:
Antecedents Process Consequences
Baseline Transition Impact
9
Messick’s ‘evidential link’
Antecedents Process Consequences
‘evidential link’
10
Phase 2• Who? 6 teacher participants from
Phase 1 • Where? 5 countries: Bulgaria, Croatia,
Lithuania, Poland, Slovakia.• When? data collection period: January
- May 05• How? Computer Mediated
Communication• What? investigation of pps’ awareness
and understanding of and reactions to NGT, to help us understand how they are preparing for the arrival of NGT
11
What?• Interviews based on monthly
tracking questions • Completion of tasks designed to
familiarise participants with NGT • Interviews discussing the task
responses
12
Data collection activities Interview 1
Task sentto pps
Pps return taskResearchers
discuss responses
Interview 2
13
The Tasks• January - Features of TOEFL preparation
classes
• February - Awareness of current and future TOEFL
• March - Focus on the new writing tasks
• April - Focus on the new speaking tasks
• May - Prediction of actual content and methodology of pps’ NGT preparation classes
14
Findings
• Understanding of NGT
• Materials
• Teacher Support
15
Understanding of NGT
Teachers seem clear about the format of NGT but less clear about the actual constructs.
16
Understanding of NGT
‘I don't think the matter of accent was really addressed, though maybe those who grade the tests have an implicit understanding of how accent plays a role and will score accordingly. In preparing students for tests, it would help to have the issue of accents and how they affect scoring addressed directly in the rubric.’
17
Understanding of NGT
Conclusion
Awareness-raising requires more than providing examples of the test.
18
Materials
Teachers rely heavily on published materials.
19
Materials ‘I can’t even dare think of something like that but I suppose I’ll have to improvise. The other possibility is to reschedule the [NGT] classes to the moment when there will be materials.’
20
Materials
ConclusionsImpact will probably be affected by the textbook writers Impact of the test may well be hard to distinguish from impact of the textbook
21
Teacher Support
The test provider has supplied various forms of teacher support but they are not necessarily accessible to the teachers who need them.
22
Teacher Support
‘I just heard it mention by [the Director] … She also said it is very expensive …. I cannot afford, the center cannot afford.’
23
Teacher SupportConclusion
Careful consideration must be paid to the logistics of teacher support if it is to be effective and reach those who need it, when they need it.
24
Professionalism• Test Provider - by commissioning
impact studies
• TOEFL teachers – by participating in our study
25
For further information • About the project:d.wall@lancaster.ac.uk t.horak@lancaster.ac.uk
• About NGT:http://toeflpractice.ets.org
26
27
Tracking questions• Have you learnt anything about New TOEFL
which you didn’t know last month? If so, what?• Is the new exam being discussed in your
institution (e.g. by other teachers/ management)? If so, what are the main issues being discussed?
• Have students asked about the new TOEFL? If so, what have they asked about?
• Are you worried/ concerned about anything concerning the new TOEFL? If so, what?
• Is there anything else of interest concerning new TOEFL which has happened this month?
top related