design of a speaking test

Universidad Católica de la Santísima ConcepciónFacultad de Educación Departamento de Lenguas Language Assessment

Unit Three assignmentSpeaking test

Names: Daniel GallardoGerardo Valdivia

Professor: Roxana Correa


SPEAKING TEST

Part 1 (2-3 minutes) (30% final mark)

Interlocutor:

A/B Good morning/afternoon/eveningCan I have your mark sheets, please?

A/B I’m……………… and this is …………………He/she is just going to listen to us

A Now, what’s your name?Thank you

B And, what’s your name?Thank you.

B Tell about your favourite vacationWhere did you go?Who did you go with?What did you enjoy the most?

Thank you

A: tell about your favourite vacationWhere did you go?Who did you go with?What did you enjoy the most?

Thank you


Part 2 (2-4 minutes) Speaking test (Trip to Australia) (70% final mark)


Interlocutor To both candidates:

I’m going to describe a situation to you:An aunt of yours is going to travel to Australia in a touristic trip. Talk together about the things she will need to take to Australia keeping in mind that is a touristic trip, and decide which are the most important things to take with her.Here you have some picture with some ideas to help you.

Show the pictures on the whiteboard (Camera, maps of the cities, bathing suit, books, pocket dictionary, travel guide), students can come up with more examples of things.

I’ll repeat the situation again:

An aunt of yours is going to travel to Australia in a touristic trip. Talk together about the things she will need to take to Australia keeping in mind that is a touristic trip, and decide which are the most important things to take with her.

(Adapted from PET Handbook, Sample Papers (Paper 3, Speaking) 2004, p. 60)


Luoma’s Specifications (2004)

The test’s purpose:

The test’s purpose is to measure vocabulary, fluency, intonation, pronunciation and grammar accuracy. Also to respond to questions about experiences of the past. To use functional language to discuss alternatives, make recommendation and negotiate agreement.

Description of the examinees: Students of 1st grade of a semi-private high school.

Test level: PET, LEVEL B1

Definition of construct (Theoretical framework for the test):

In education there are many important factors to take into account in order to make it an affective process. This process embraces from the delivering of the content to the students by the teacher, to the part of the process in which the students show how much they have learnt. Even though many people believe that assess is the easiest part of the process, which is not true. To make a good and effective assessment it is important to consider many factors. For example, it is important to know that the assessment must be coherent with the content that has been seen in classes, this means that when you plan the lessons it is important to make the subject content coincide with the content requested in the evaluation. If the teacher does that, the assessment will be valid.

Even though there are many skills, the speaking skills, in terms of learning a second language is one of the most important. Orwig (1999) describes the concept of Speaking as the productive skill in the oral mode, and he establishes that it is, like the other skills, more complicated than it seems at first, and declares that it is not just about pronouncing words.

Luoma (2004) also gives her opinion about the speaking skills. Luoma (2004) describes the speaking skills as an important part of the curriculum in language teaching, and that this makes them an important object of assessment as well. Luoma (2004) says that assessing speaking is challenging. Some reasons that support her statement is that there are many factors that influence our way of analyzing how well


someone can speak a language, and teachers constantly expect their test scores to be accurate and appropriate for their purposes, but they not always succeed on that.

Luoma (2004) presents some important issues that are necessary to take into account at the time of assessing speaking. Luoma (2004) also expresses that it is necessary to make students understand what they are going to do. She also explains that the teacher must guide the examinees to talk about what he is going to assess. To do this, the teacher must outline the content to be assessed, and also provide a context to it. Is in this part where we get into the concept of construct. Construct is referred to the moment in which the task is designed. Talking about assessing speaking, Luoma (2004) describes an important factor called the construct-related information that the scores must deliver, or in other words what the score users need to know about the examinees´ speaking skills.

Luoma (2004) expresses that the teacher must outline the content to be assessed and also provide a context to it. The word construct is referred to the time in which the task is designed. In this part of the process the teacher take into account the skills he wants to measure in students. He must to see what their students are going to do, taking into account the skills they are going to use to develop the task, and finally, with that information create a rubric to assess them. What this dictates is the types of skills that the tasks should make the examinees show in learning-related assessment. Even though this is often guided by what have been taught recently, tests may need to provide more general information about the examinees´ skills, more or less like formal, external examinations.

Description of suitable language course or textbook:

This task was taken and adapted from The Cambridge English: Preliminary (preliminary English Test- PET). We believed that this task was according to level of English of the students that it was made for. We also believed that it was a clear example of construct, showing the relation between the task and the descriptors presented in the rubric, making them coherent.

Number of sections/papers: 2


Time for each section/paper:

Part Task Timing Weighting

1 Each candidate interacts with the interlocutor.The interlocutor asks the candidates questions in turn, using questions.

2-3 minutes 30%

2 Simulated situation. Candidates interact with each other.Visual prompts are given to the candidates to aid the discussion task. The interlocutor sets up the activity using a rubric.

2-4 minutes. 70%

Target language situation: Students will be able to exchange information or opinions about what to take in a touristic trip, orally.

Text-types: Pictures

Text length is 6 pictures of element you can take to a touristic trip to Australia.

Language skill to be tested is speaking

Language elements to be tested are Fluency, grammar accuracy, turn taking, intonation

Test task: Part 1 of the test tasks consists in exchanges with the examiner and part 2 is a collaborative task involving both candidates.

Test methods: Part 1

The test begins with a conversation led by the interlocutor, who asks the candidates questions about vacations they have had. Candidates speak in turn and are not expected to talk to each other at this stage. The purpose of this conversation is to test the language and to enable each candidate to overcome any initial nervousness. The interlocutor’s questions are designed to elicit long responses.

This part of the test assesses the candidates’ ability to express ideas of events of his own knowledge, the candidate may even make up the situations since the language they will use it will be assessed not the veracity of the story.


Part 2

This part is about a simulated situation where the candidates are asked to make and respond to suggestions, discuss alternatives, make recommendations and negotiate agreement with their partner, related to a touristic trip of a relative. In this part of the test, the candidates speak to each other.

The interlocutor sets up the task and repeats the instructions while candidates look at the prompt material on the whiteboard. The interlocutor then takes no further part in the interaction. In the event of a complete breakdown in the interaction, the interlocutor may subtly intervene to redirect the students, but will not take part in the task itself. Candidates are expected to engage with the task independently, negotiating turns and eliciting opinions from each other.

On the whiteboard some visual prompts are given to the candidates which is designed to generate ideas and provide the basis for the discussion. Candidates may, however, introduce their own ideas if they wish. Candidates are assessed on their ability to take part in the task, so it is not necessary for them to complete the task in the time given. Candidates are assessed on their use of appropriate language.

Rubrics:

The rubrics used in the test are rating scales. For part 1 the rubric has 20 points in total because the “turn taking” dimension is not present since the task consist in open questions, so there is no turn taking present, and the rating is from 4 - 1 in 5 different criteria. For part 2 the rubric has 24 points in total because the dimension “turn taking” is added and the rating is from 4 - 1 in 5 different criteria. The rating scales in the highest level of performance express the idea of a student who is able to use vocabulary and expression correctly, who can make occasional grammatical errors, who speaks with little hesitation that does not interfere in the message, who is cohesive and coherent with what s/he says, who answers the questions and develops interaction, that his/her pronunciation and intonation is clear and accurate, and that is capable of use his/her turn appropriately. The rating scale in the lowest level of performance suggests that the student is only able to use basic expressions and vocabulary, makes frequent grammatical errors, hesitates to often, is not able to communicate, does not respond appropriately, has frequent problems with pronunciation and intonation and that is not able to use his/her turn appropriately.


Speaking Rubric Part 1 (30%)

Name: _________________________ Rating:___/20 Activity: Open questionsComments:_________________________________________________________________________________________________________________________________

Rating Dimension Demonstrated competence

Excellent

4

Use of vocabulary

Uses a variety of vocabulary and expressions

Grammar Uses a variety of structures with only occasional grammatical errorsFluency Speaks smoothly, with little hesitation that does not interfere with

communicationContent Stays on task and communicates effectively; almost always responds

appropriately and always tries to develop the interactionPronunciation Pronunciation and intonation are almost always very clear/accurate

Good

3

Use of vocabulary

Uses a variety of vocabulary and expressions, but makes some errors in word choice

Grammar Uses a variety of grammar structures, but makes some errorsFluency Speaks with some hesitation, but it does not usually interfere with

communicationContent Stays on task most of the time and communicates effectively; generally

responds appropriately and keeps trying to develops the interactionPronunciation Pronunciation and intonation are usually clear/accurate with a few

problem areas

Adequate

2

Use of vocabulary

Uses limited vocabulary and expressions

Grammar Uses a variety of structures with frequent errors, or uses basic structures with only occasional errors

Fluency Speaks with some hesitation, which often interferes with communication

Content Tries to communicate, but sometimes does not respond appropriately or clearly

Pronunciation Pronunciation and intonation errors sometimes make it difficult to understand the student

Inadequate 1

Use of vocabulary

Uses only basic vocabulary and expressions

Grammar Uses basic structures, makes frequent errorsFluency Hesitates too often when speaking, which often interferes with

communicationContent Purpose isn’t clear; needs a lot of help communicating; usually does

not respond appropriately or clearlyPronunciation Frequent problems with pronunciation and intonation

(Adapted from Authentic Assessment for English Language Learners by J. Michael O'Malley and Lorraine Valdez Pierce, Addison-Wesley Publishing Company)


Speaking Rubric Part 2 (70%)

Name: ______________________ Rating:___/24 Activity: Collaborative taskComments:_________________________________________________________________________________________________________________________________

Rating Dimension Demonstrated competence

Excellent

4

Use of vocabulary

Uses a variety of vocabulary and expressions

Grammar Uses a variety of structures with only occasional grammatical errorsFluency Speaks smoothly, with little hesitation that does not interfere with

communicationContent Stays on task and communicates effectively; almost always

responds appropriately and always tries to develop the interactionPronunciation Pronunciation and intonation are almost always very clear/accurate

Turn taking Speaker is able to speak and lets the other speaker to develop his ideas

Good

3

Use of vocabulary

Uses a variety of vocabulary and expressions, but makes some errors in word choice

Grammar Uses a variety of grammar structures, but makes some errorsFluency Speaks with some hesitation, but it does not usually interfere with

communicationContent Stays on task most of the time and communicates effectively;

generally responds appropriately and keeps trying to develops the interaction

Pronunciation Pronunciation and intonation are usually clear/accurate with a few problem areas

Turn taking Speakers are able to make the communication flow and sometimes they overlap the other speaker’s turn to speak.

Adequate

2

Use of vocabulary

Uses limited vocabulary and expressions

Grammar Uses a variety of structures with frequent errors, or uses basic structures with only occasional errors

Fluency Speaks with some hesitation, which often interferes with communication

Content Tries to communicate, but sometimes does not respond appropriately or clearly

Pronunciation Pronunciation and intonation errors sometimes make it difficult to understand the student

Turn taking Speaker overlaps the other making the conversation difficult to followInadequate 1

Use of vocabulary

Uses only basic vocabulary and expressions

Grammar Uses basic structures, makes frequent errorsFluency Hesitates too often when speaking, which often interferes with

communicationContent Purpose isn’t clear; needs a lot of help communicating; usually does

not respond appropriately or clearlyPronunciation Frequent problems with pronunciation and intonation

Turn taking Speaker does not respect his turn to speak

(Adapted from Authentic Assessment for English Language Learners by J. Michael O'Malley and Lorraine Valdez Pierce, Addison-Wesley Publishing Company)


Fulcher’s specifications The Task Orientation is opened since the students are given some information, but the information they can use it as they wish or they may even come up with more information. They can use the words they want and decide what tenses to use. Obviously the tenses and words have to be well-used because the purpose is to speak properly.

Interactional relationship: the test in part 1 is a one way interaction; since the interlocutor asks some questions and the examinees have to answer them. The part 2 is a two ways interaction because the examinees have to construct a conversation to get to an agreement about what a relative have to take with her in a trip to Australia.

Goal orientation: In part 1 of the test there is not a goal orientation since the examinees only answer questions, this part does not have a communicative purpose. While in part 2 there is a goal interaction since examinees have to talk to each other to get to an agreement

Interlocutor status and familiarity: since there are two examiners (interlocutor 1 and interlocutor 2) in the test with the one who is only listening the interlocutor status is high. With the interlocutor who the examinees interact with the status is balanced, because students know the examiner who may be their English teacher. It can be added that the interlocutor status is balanced and not low because a test is a formal situations, so

The familiarity with Interlocutor 2 is low because the examinees do not know this examiner. The familiarity with Interlocutor 1 is high since they know each other.

Topics: travelling abroad.

Scoring procedure: Part 1= 30% + Part 2= 70% (P1 x 0.3) + (P2 x 0.7) = X


References

Downes, S. (2004). PET Handbook, Sample Papers (Paper 3, Speaking) [PDF file]. (p.

60). Available from http://www.iltea.org/cambridge/PET/PET_HB_sampleS.pdf

Fulcher, G. (n.d.) Examples of Task Types. Retrieved on June, Tuesday 2nd, 2014 from http://languagetesting.info

Luoma, S. (2004). Assessing Speaking. Cambridge: Cambridge University Press.

O’Malley, J. M., & Valdez-Pierce, L. (1996). Authentic Assessment for English Language Learners. USA: Addison-Wesley Publishing Company.

Orwig, C. J. (1999). Prepare for language learning.

http://www.iltea.org/cambridge/PET/PET_HB_sampleS.pdf

design of a speaking test

Education

wesley publishing company

word choice grammar

generally responds appropriately

lorraine valdez pierce

communication content stays

english language learners

pronunciation frequent problems

interaction pronunciation pronunciation