1

Task-based Assessment of Group-constructed Oral Interaction

in EFL Classrooms

Presentation forThe 3rd Biennial Conference on Task-based Language Teaching

by

Angie H.C. Liu, Ph.D. (*angiehcliu@cycu.edu.tw)Department of Applied Linguistics and Language Studies Chung Yuan Christian University

September 13-16, 2009

2

Introduction

＊ Group-based oral discussions activate LLs’ interaction skills --

-- distributing and competing for opportunities to speak,

-- holding the floor,

-- accommodating other speakers,

-- making on-line adjustments.

3

Introduction

＊ format and design of group-based oral assessment places all test takers in equal power positions in relation to each other

brings out the kind of interaction that is not possible via traditional one-to-one oral interviews

4

Introduction

＊ test discourse -- jointly created by multiple test takers, potential influential parameters

-- characteristics of individual test takers,

-- group dynamics and composition,

-- assessment tasks,

-- scoring criteria

5

Introduction

＊ reality -- teachers often avoid group oral assessment concerns related to the scoring of such a complex and dynamic construct

＊ issues of validity, reliability and fairness

＊ traditional psychometric framework focuses on individual performance

6

Introduction

＊ The current study investigates –

impact of differential scoring approaches (i.e., group-scoring vs. individual-scoring) on the group-based oral interaction in assessment context

7

Introduction

＊ This is accomplished by analyzing EFL

test takers’ interaction profiles elicited

from collaborative assessment tasks under

different scoring approaches.

8

Research Question

Specifically, the following question is being examined:

“To what extent does the scoring approach affect EFL test taker’s oral interaction profiles elicited via collaborative tasks?”

9

Study Design

＊ Participants

-- 48 EFL college freshmen from two intact classes

-- their English proficiency level (intermediate)

10

Study Design

＊ Procedures

1. Each participant was randomly assigned to a test group of four.

2. Each test group was randomly assigned to one of the two oral assessment tasks.

3. Each group was allotted 10-minute pre-task planning time.

11

Study Design

4. Prior to the planning time, participants had been informed of the scoring method (i.e., group- or individual-based) that would be used to evaluate their oral performance.

-- a counter-balanced test administration procedure was implemented to avoid potential interaction between assessment tasks and scoring methods (see table 1).

12

Study Design

Table 1: test administration plan

Class 1 Class 2

Task 1: Individual Scoring

Task 2: Individual

Scoring

Task 2: Group

Scoring

Task 1: Group Scoring

13

Instrument and Material

＊ Attributes of the Oral Assessment Tasks: -- require test takers to discuss their reactions to

hypothetical crises (e.g., ship rescue) -- non-convergent tasks -- all members of the test group received exactly the same information

14

Instrument and Material

＊ Task 2: as a group, discuss what you will do in the following scenario:

“ A fierce storm hit two passenger cruises, which started to sink. Both ships carry the same number of passengers. You are the leader of the coast guards, who came to the rescue. Unfortunately, your crew and you can only rescue one ship at a time. The second ship to be rescued would have for sure sunk to the bottom of the ocean by the time you get to them.”

15

Instrument and Material

＊ Rating scale: -- a ten-point scale was used for evaluating individual

and group oral performance

-- rating criteria include fluency, content, communication and interaction

-- holistic scores were produced

16

Data Analysis

• The interaction profiles of EFL test takers were analyzed in terms of

-- discussion quantity

-- turn-taking patterns

-- repair type

17

Results and Discussion

• Discussion Quantity (see Table 2)

Analysis Criteria Individual Scoring Group Scoring

Total word count 872 885

Unduplicated content word count

342 (39.2%)

322 (36.4%)

18

Results and Discussion

(see Table 2)

1. ELL participants were found to generate similar amount of discussion regardless the scoring method used (872 word/IS vs. 885 word/GS).

19

Results and Discussion

2. A slightly higher proportion of unduplicated topic-related content words was identified when individual scoring was used (39.2%/IS vs. 36.4%/GS).

20

Results and Discussion

3. When GS was used, members of the same test group tended to repeat or respond to the same ideas by using more discourse markers such as “I think”, ‘mm-hm’, ‘er’, and ‘yeah’.

4. When IS was used, individual test members tended to initiate new ideas.

21

Results and Discussion

Turn Taking In this study,

* a turn refers to “an actual occurrence of holding the floor”

–> turns were counted when transfer of speakers occurred at transition-relevance points

22

Results and Discussion

* Turn allocation was examined in terms of

1) the number of turns claimed by each speaker

2) the selection of next speaker

23

Results and Discussion

Table 3.1

Analysis Criteria IS GS

Number of turns exchanged 116 135

Length of turns (*mean word count per turn)

7.5 6.6

Turn-allocation variance 2.9 3.2

24

Results and Discussion

1. More turns were exchanged among test members when GS was used.

2. The duration of turns was shorter when GS was used.

25

Results and Discussion

3. As shown by the turn-allocation variance, floor dominance by few test takers was more apparent when IS was used.

26

Results and Discussion

* Turn-allocation pattern –

Results showed that the overall pattern of speaker selection was quite similar between the two scoring methods.

27

Results and Discussion

Table 3.2 Turn Allocation Pattern

Analysis Criteria IS GS

Turn-allocation pattern

selected by current speaker 4 (3.8%)

3 (2.6%)

self-selected speaker 60 (56.6%)

62 (53.9%)

no one takes up the turn so the current speaker continues

42 (39.6%)

50 (43.5%)

28

Results and Discussion

* Specifically,

1. When the floor was perceived to be open for grab, EFL test takers tended to self-select themselves as the next speaker.

29

Results and Discussion

2. The second most frequently adopted option for speaker selection was for the current speaker to continue when no one takes up the turn.

30

Results and Discussion

3. It turned out that EFL test takers rarely named the next speaker for a turn in the assessment context.

-- inconsistent with the speaker-selection pattern

revealed in conversations carried by EFLs in

non-testing context.

31

Results and Discussion

Repair

* A repair is an attempt to address the problematic communication.

It serves as a vital mechanism for maintaining reciprocal interaction between speakers.

32

Results and Discussion

Table 4

Analysis Criteria IS GS

Total count of repair 8 4

Type of repair

self-initiated self-repair 6 3

other-initiated self-repair 1 1

self-initiated other-repair 1 0

other-initiated other-repair 0 0

33

Results and Discussion

1. Not many instances of repairs were identified in this study.

2. Among them, self-initiated self-completed repairs occurred most frequently and relatively more frequent when IS was used.

34

Results and Discussion

3. As the goal for task-based interaction is to

accomplish the task in hand, repairs focus on

establishing mutual understanding between speakers.

-- this study showed that incorrect linguistic forms and interlanguage forms were frequently ignored by test takers unless they lead to complete communication breakdown.

35

Results and Discussion

4. The correction of linguistic errors, if made at all, was mostly done via self-initiated self-completed repairs.

-- consistent with Seedhouse’s claim (2004) that ‘a learner in learner-learner interaction never attempts to correct another learner’s linguistics forms in task-oriented contexts.’

36

Conclusions

Key Findings:1. The two scoring approaches resulted in similar quantity of group

discussions (*total number of words) .2. Turn-taking was more equally distributed among group members

when group-scoring was used.3. More content information and instances of self-initiated self-

completed repairs were identified when individual scoring was used.

37

Conclusions

• To summarize,

it appears that the choice of the scoring approach exerts real influence on the assessment outcome of task-based group oral interaction in EFL classrooms.

38

Conclusions

• When the performance is viewed as a joint product, EFL test takers show higher collaboration effort and focus more on meaning-based exchanges.

39

Conclusions

* Pedagogical Implications

1. If the goal is to develop the interactional competence of language learners, it is recommended that instructors assess them on a group basis, ignoring differential contributions from individuals.

40

Conclusions

2. It seems that task-based group oral testing is more effective in evaluating EFL learners’ development in the meaning-and-fluency aspect rather than the form-and-accuracy aspect.

ReferenceSeedhouse, P. (2004). The Interactional Architecture of the Language Classroom: A

Conversation Analysis Perspective. MA: Blackwell Publishing.