11/15/2016

1

WASH ı E LT SS T E P BACK

M E H TA P I N C E , I S TA N B U L Ş E H I R U N ı V E R S ı T YT U R K E Y

PAıR DıSCUSSıON

• Does your institution give an in-house or an internationally recognized test to assess students’ proficiency? Why? What about your preference?

• What do you know about the content and specifications of IELTS?

11/15/2016

2

READINGCOMPONENT

3 texts of different genres and lengths, from journals, books, magazines and newspapers

Total: 40 questions, 60 minutes

Question Types: MC,T/F/NG,Yes/No/NG, Matching, Sentence/ Summary/Table/Diagram Label Completion, SA

LISTENINGCOMPONENT

While Listening: 4 sections, each with 10 questions, audio heard onceThe first two are related social contexts; a conversation btw 2 & a monologueThe last two are related to academic contexts; a conversation btw 4 & a monologue

Total: 40 questions , 45 minutes (10 minutes for transferring answers)

Question Types: MC,T/F/NG,Yes/No/NG, Matching, Sentence/ Summary/Table/Diagram Label Completion, SA

WRITINGCOMPONENT

Task 1: 150 words, 20 minutes, description of a process or a diagramTask 2: 300 words, 40 minutes, presenting a solution to a problem/ justifying an opinion/ evaluating ideas and evidence

SPEAKINGC0MPONENT

Part 1: Interview with the examiner, personal questions, about 4 minutes

Part 2: A task card, 1 min. for preparation, monologue, about 1 min, followed by more questions on the same topic

Part 3: The examiner and examinee discuss more abstract topics linked with the Part 2 questions, about 4 minutes

Total: about 11-14 minutes

ıELTS

11/15/2016

3

READINGCOMPONENT

SEHIR UNIVERSITY TEST OF ENGLISH PROFICIENCY (STEP)

READING TEXT 1: 15%, about 1100 words, 10 MC questions

READING TEXT 2: 15%, about 1500 words, 10 MC questions

Total: 60 minutes

LISTENINGCOMPONENT

Listening 1 (While Listening): 10%, 10 MC questions, 10 minutes, audio heard once

Listening 2 (Note-taking): 15%, 10 MC questions, 25 minutes, audio heard once

Total: 35 minutes

WRITINGCOMPONENT

Argumentative Essay Writing: 25%, 60 minutes, 300+ words

SPEAKINGCOMPONENT

Part 1: Interview with the examiner, personal questions, about 2 minutes

Part 2: Monologue, answering an academic question for about 1 minute, about 4 minutes

Part 3: Discussion with the other examinee, a discussion prompt, about 8 minutes

Total: about 15 minutes

11/15/2016

4

11/15/2016

5

T H ı N KD ı S C U S SR E F L E C T

S H A R E

CHOOSE A HAT FOR YOUR HEAD

11/15/2016

6

DıSCUSSıON QUESTıONSSTUDENTS TEACHERS MID-ADMIN

1. How do you think students have felt about this shift from IELTS to an in-house test?

1. How do you think teachers have felt about this shift from IELTS to an in-house test?

1. How do you think levelcoordinators and testing membershave felt about this shift from IELTS to an in-house test?

2. Do you think there have been any changes in their attitudes, motivation, or learning/studying styles and habits?

2. Do you think there have been any changes in their lesson planning, teaching materials, and teaching methods? If yes, what changes have happened?

2. Do you think they have needed to make any changes in the curriculum and test specifications?

3. Which one do you think students would prefer, STEP or IELTS? Why?

3. Which one do you think teachers would prefer, STEP or IELTS? Why?

3. Which one do you think mid-admin group would prefer, STEP or IELTS? Why?

PURPOSE OF THE STUDY

• to observe different washback patterns among teachers, students, and mid-admin group; level coordinators and test writers

• to identify how the new in-house test is perceived

• to support innovations in curriculum and assessment within the program, through looking at the issue with the eyes of all the parties involved as much as possible

11/15/2016

7

PARTıCıPANTS AND PROCEDURE

Interview

7 experienced teachers in the exit level

Questionnaire

Teachers who have taught the exit level, at least once before

and once after

Students who have taken both IELTS and the new in-house exit

test

Mid-admin group: level coordinators, level test writers and STEP test

writers

Informal Interview

The exit level (PFAC) coordinator and an experienced instructor in the exit level

2 experienced test writers of the in-house exit test (STEP)

11/15/2016

8

•

11/15/2016

9

• Students are happy as«... guided well and they think STEP is more achievable due to MC questions,....it’s their perception.»« they can object and get feedback.»«their success rate is higher and they are less afraid.....its content is more relevant.»«they are not looking for outside exam oriented courses and they’re aware they need to improve their English to pass»• Students are not happy as «they are now panicked. They used to feel more comfortable with IELTS.»

• Admin people are happy because«students used to get stuck with IELTS more.»«they now have their own high stakes exam. It is prestigious.»

• Instructors are happy because «they now have more self-dependent learners. Students try to meet program expectations.»«they can plan their lessons more competently as STEP is more predictable and familiar to them...»«IELTS used to confuse me. I teach Academic English but IELTS assessed what I did teach. I used to feel I had to teach to the test, which made my lessons more exam oriented. I feel more flexible.«.... Readily accessible materials lack. They don’t complain much, though.»

11/15/2016

10

• «Since IELTS is gone, students' chances are more limited now. So, they tend to have much more exam anxiety.

• «Instructors also have more duties.»• «The only drawback for our level is preparing extra materials.»• «..faculties all demand... It's unfair.»

*«shared responsibility lacks in teachers; see grading and invigilation as extra work»*«ss’ pressure on admin and us»*«security of tests and personal computers» *«ss’ objections and asking for feedback»*«worries about whether the level is adjusted to the standards»*«fear of covering some items/objectives in class while teaching»

• «The program is not bound to the IELTS calendar any more....more flexibility....»• «All respectable schools have their own proficiency.. this shows the degree of quality...»• «In terms of practicality, it's better(cutting down on expenses, flexibility with timing and objectives).»• «...students' reactions.They used to demand IELTS practice to do in class. However, they now

demand academic reading texts, note-taking exercises and academic writing feedback.»• «better suited exam considering the needs of our program and departments»

*«It is highly motivating to engage in a new and challenging process.»*«better analysis of students’ profile and needs, test specs based on principles of already existing curriculum»*«active involvement of testers in curriculum innovations»*«less room for surprises»*«high face validity»*«testers are not outsiders»

11/15/2016

11

REFLECTıONS & ıMPLıCATıONS• The new in-house exit test has encouraged more in-depth analysis of students’ strenghts and

weaknesses. Reading booklets and reading quizzes have been integrated.

• Upper levels have started emphasizing academic tone of writing more and added sections to their booklets.

• Note-taking listening skills, interaction skills and critical thinking skills assessed in speaking, academic tone of voice in writing, more emphasis on academic skills in both writing and speaking criteria

• Sense of ownership among admin, mid-admin people, and instuctors

• Inevitable journey from ‘validity over reliability’ to ‘practicality over validity’.

• «.. to avoid negative washback from exam preparation, teachers should be aware of the exam requirements. They need to have a sound knowledge of the aims, specifications and administrative procedures of the exam they are preparing for.» Tsagari

GOıNG FURTHER – FOLLOW-UP

Replicating the study a year later, integrating student

interviews and class observations

Disseminating the results within and outside the institution

Comparing the percentage of students completing the program but failing the in-house exit test

and IELTS

Searching ways to enhance teachers’ and students’ literacy skills (self and

peer-feedback) because this may help positive washback, more than a reliable

and a valid test can

Discussing the results with both admin and mid-admin people to raise their awareness of how the exit test could

encourage innovations in the curriculum without dominating it

11/15/2016

12

YOUR CONTRıBUTıONS

REFERENCESAlderson, J. C. & Hamp-Lyons, l. (1996). TOEFL preparation courses: a study of washback. Language Testing. 13, p. 280-297. Retrieved from: https://www.researchgate.net/publication/240738514

Cheng, L. (2014). Consequences, Impact, and Washback. Retrieved from: https://www.researchgate.net/publication/270703143_Consequences_Impact_and_Washback

Cheng, L. , Watanabe, J. & Curtis, A. (Eds.) (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum.

Green, A. B. (2007). IELTS washback in context: Preparation for academic writing in higher education.

Feast, V. (2002). The Impact of IELTS Scores on Performance at University. International Education Journal, 3(4). Retrieved from: http://ehlt.flinders.edu.au/education/iej/articles/v3n4/feast/paper.pdf

Pan, Y. (2008). A critical review of five language washback studies from 1995-2007: Methodological considerations. SHIKEN: JALT Testing & Evaluation SIG Newsletter. 12 (2). p. 2-16. Retrieved from: http://jalt.org/test/PDF/Pan1.pdf

Tsagari, D. (2011) Washback of a high-stakes English exam on teachers’ perceptions and practices. Retrieved from: http://www.enl.auth.gr/symposium19/19thpapers/042_Tsagari.pdf