international journal of language learning and applied ...ijllalw.org/finalversion5134.pdfbecome as...

International Journal of Language Learning and Applied Linguistics World (IJLLALW)

Volume 5 (1), January 2014; 419-‐440 Birjandi, P., & Azad, M ISSN (online): 2289-‐2737 & ISSN (print): 2289-‐3245 www.ijllalw.org

419

A COMPARATIVE EXPLORATION ON THE EFFECTIVENESS OF DIFFERENT APPROACHES TO DYNAMIC ASSESSMENT

OF LISTENING COMPREHENSION IN THE CONTEXT OF IRAN

Parviz Birjandi, PhD

Science and Research Branch, Islamic Azad University (IAU), Tehran, Iran Email address: [email protected]

Mahnaz Azad, PhD Candidate

Science and Research Branch, Islamic Azad University (IAU), Tehran, Iran Email address: [email protected]

ABSTRACT Listening ability is one of the important skills in foreign language learning. In spite of its importance, listening has long been the neglected skill in foreign language acquisition, research, teaching, and assessment. There has been little research on the listening problems that English as a Foreign Language (EFL) or English as a Second language (ESL) learners experience in learning to listen to spoken English in the classrooms. The present study aimed to conduct a comparative investigation on the diagnostic capacities of three different approaches to Dynamic Assessment (DA), namely, interventionist and interactionist, as well as the potential effects of multi-media DA- based instruction on the development of listening comprehension ability of Iranian university students studying Teaching English as a foreign language (TEFL). The study was based on a longitudinal research design enabling the observation of listening development over time. Based on these premise, the study implemented a mixed methods research methodology, i.e., microgenetic analysis, a key sociocultural method, and proposition analysis, as a supplementary analytical method. The results of the study showed that lack of effective listening strategies, lack of functional knowledge, insufficient phonological knowledge, deficient vocabulary and grammatical knowledge, together with many other factors were the major listening comprehension problems encountered by EFL Iranian learners. Understanding students' learning difficulties may enable EFL teachers to help students develop effective learning strategies and ultimately improve their English listening skill. Suggestions are made for addressing problems regarding how teachers can help their students overcome listening comprehension problems. The results of this study may also be useful for those who are interested in this field. KEYWORDS: dynamic assessment, interventionist, interactionist, multimedia, CDLT INTRODUCTION Listening ability is one of the important skills in foreign language learning. In spite of its importance, listening has long been the neglected skill in foreign language acquisition, research,



420

teaching, and assessment. There has been little research on the listening problems that EFL/ESL learners experience in learning to listen to spoken English in the classrooms. On the other hand, despite the fact that DA is gaining increased attention in applied linguistics all over the world, empirically grounded research scrutinizing the effects of DA on L2 acquisitional processes still remains scarce. Considering the premise, the present study was an attempt to conduct a comparative investigation on the diagnostic capacities of three different approaches to DA, namely, interventionist and interactionist, as well as the potential effects of multi-media DA- based instruction on the development of listening comprehension skill of university students majoring in TEFL. The participants of this study were nine female students studying at Islamic Azad University, Tehran North Branch. This study applied the aforementioned three DA-based approaches applying a longitudinal research design to enable the observation of listening development over time. Based on these premises, the study implemented a mixed methods research methodology, i.e. microgenetic analysis, a key sociocultural method, and proposition analysis, as a supplementary analytical method striving to add a new dimension to L2 DA research as well as to L2 research and pedagogy by applying an innovative approach, i.e., comparing the three approaches to DA, to investigate, diagnose, and promote listening development in an EFL context. REVIEW OF THE RELATED LITERATURE DA is a relatively new approach to L2 assessment that has been introduced to L2research and educational community by Lantolf and Poehner (2004) and Poehner and Lantolf (2005). Since 2004, there has been growing support for the use of DA in language pedagogy (Ableeva, 2007, 2008; Antón, 2009; Lantolf & Poehner, 2004; Poehner & Lantolf, 2005; Poehner, 2005, 2008). The authors of L2 publications on DA argue in favor of this qualitative procedure and provide examples of how students benefit from DA-based language instruction. By definition, Dynamic assessment integrates assessment and instruction into a seamless ,unified activity aimed at promoting learner development through appropriate forms of mediation that are sensitive to the individual's (or in some cases a group's) current abilities. In other words, contrary to traditional assessment that focuses on already matured abilities, "DA promotes functions that are maturing" in the ZPD and "foregrounds future development" (Lantolf & Poehner, 2004, p. 54). These manifestations of DA result from its underpinning in such foundational SCT concepts as the zone of proximal development, mediation (or collaboration, in Vygotsky's terms), and learners' responsiveness to mediation. That is, SCT-based DA sees mediation as a pedagogical instrument that "is provided during the assessment procedure and is intended to bring to light underlying problems and help learners overcome them" (Lantolf & Poehner , 2008, p. 273). Following Vygotsky, L2 research views DA as a procedure that unifies assessment and instruction (e.g., Lantolf & Poehner 2004, 2008). It is argued that assessment and instruction are inseparable components of the same dialectical activity and that "assessment and instruction become as tightly conjoined as two sides of the same coin and there are no one-sided coins" (Lantolf & Poehner, 2008, p.274). This point of view is in sharp contrast to traditional testing procedures that dichotomize instruction and assessment and conceive assessment in a more



421

conventional, standardized way (Haywood & Lidz, 2007; Poehner, 2005; Sternberg & Grigorenko, 2002). According to Lantolf and Poehner (2004), DA comprises Interventionist and Interactionist types that usually involve three stages: pre-test → mediation → post-test .The most important proponents of these two orientations are Budoff and Feuerstein respectively. The interventionist type of DA includes intervention from the examiner during the test procedure itself but it is a more formal and standardized approach. During interventionist DA, the examinees are given instruction item by item and if they cannot solve the item correctly, they are given pre-fabricated hints. Interactionist DA entails mediation emerging from interaction between examiner and examinee. During interactionist DA, leading questions, hints or prompts are not planned in advance; instead, they emerge from mediated dialogue (or collaborative interaction) between the examiner and the examinee in which the examiner reacts to the examinee's needs and constantly re-calibrates his/her mediation. It is important to note here that within DA the examiner-examinee relationship is based on the idea of teaching and helping, e.g. learners are allowed to pose questions and receive immediate feedback .Within both formats of DA, the instruction may be given in individual or group settings (Poehner & Lantolf, 2005; Poehner, 2005; Sternberg & Grigorenko, 2002). Although at present DA is attracting increased attention among applied linguists and L2 educators, empirical research on DA-based instruction in second and foreign language acquisition remains scarce (see Ableeva, 2008; Antón, 2009, Poehner, 2005). However, the few DA-based studies done so far (e.g., Ableeva, 2008) suggest that DA is a valuable qualitative tool which allows instructors to diagnose the actual level of students‘listening abilities, as well as those abilities "that are now in the state of coming into being, that are only ripening, or only developing" (Vygotsky, 1986, pp. 447-8). The promising results from these studies served to motivate the present investigation. Multimedia Listening Quite a bit has been written about the different characteristics of multimedia that can enhance reading (Chun & Plass, 1997), vocabulary acquisition (Plass, Chun, Mayer & Leutner, 1998), and even speaking (Borras & Lafayette, 1994). However, a smaller number of studies (Brett, 1996, 1997) investigated the use of multimedia software for listening comprehension. When investigating learners’ attitudes towards multimedia, Brett (1996) found that 86.9% of students believed that a multimedia application they were introduced to would improve their listening skills. Also, in questionnaires, participants preferred multimedia for listening over media such as video and audio (Brett, 1997). Finally, learners had better listening comprehension scores on a multimedia task than on a paper and pencil task in which the input was delivered via video and audio tapes (Brett, 1997). More research into the use of multimedia listening materials is necessary as is research into help options, which Pujolà (2002) defined as "resources of the program which assist the learner in performing a task" (p. 241). Many theories have been proposed in this regard, one of which is interactionist theory of CALL (Long, 1996; Pica, 1994).



422

A key component of this theory, that only the input that is noticed, or apperceived, can become beneficial, provides guidance for the design of instructional materials, which should contain features that enhance input through modifications (Chapelle, 2003, p. 40) such as added redundancy and change of the input mode. Following Chapelle’s (1998) suggestions relevant for the development of multimedia CALL, this study is to investigate the effectiveness of two types of input modifications, i.e., ‘transcript’ and ‘useful expressions’ options, within some listening comprehension tasks. In particular, this textual help option is selected because it can add redundancy to the aural input by changing the input mode from its aural form in the video into the textual form of subtitles. Moreover, participants will have the advantage to use an online English- English dictionary whenever needed. RESEARCH QUESTIONS In order to investigate the study goals, the following research questions were formulated:

1. Does an interventionist DA (IV) procedure enhance the development of listening skill among EFL language learners?

2. Does an interactionist DA (IT) procedure enhance the development of listening ability among EFL language learners?

3. Does a multi-media interventionist based DA (MT) procedure enhance the development of listening skill among EFL language learners?

4. How frequently do the MT participants use the two help options offered in a multimedia activity in each group?

5. Does any specific form of mediation during DA-based pedagogical interventions best nurture the development of listening skills of EFL language learners?

6. Do the learners exhibit any specific moves to mediation during the DA-based pedagogical interventions?

METHODOLOGY This study implemented a mixed method design, applying qualitative and quantitative approaches in order to conduct the pedagogical experiment and to report its results. The use of different methods was necessary to attain a comprehensive view of the research findings. Following a SCT-based DA framework, the study gave priority to a qualitative approach which is best suited to the ZPD concept. Accordingly, a qualitative approach was applied to interpret the data obtained during the mediated portions (or learners’ mediated listening recalls) of the DA and carry over sessions. Despite a preference for qualitative analysis, and to supplement this approach, minimal quantitative analysis was also incorporated into the study in the form of frequency counts of mediation offered by the instructor and responsiveness to mediation provided by the learners. Participants The participants of the study were nine female university students whose scores were at three different levels of language proficiency selected based on ‘the Oxford Level Guide’ as follows:



423

Table 1: Biographical/ language background information of the participants

Given Name

Pre-test Listening score

Gender

Age

No. of years studying Eng.

Semester studying in university

Travel overseas

Time in English-speaking countries

How often they listen to Eng. Texts within a week

A1

57

F

35

6

2

4 Times

4

Every day, several hours

A2

58

F

27

5-6

2

3 Times

2

Every day, at least 3 hours

A3

55

F

35

5

3

Once

-

Every day, at least 3 hours

B1

44

F

28

4

2

Once

1

Almost every day, 2 hours

B2

39

F

31

4

2

None

-

4-5 times in a week, 1-2 hours

B3

37

F

33

4

2

None

-

3 days in a week, 1-2 hours

C1

22

F

31

1

2

2 Times

1

2 times in a week, At least 1 hour

C2

27

F

28

2

3

None

-

Once a week, about 1 hour

C3

24

F

25

1

3

None

-

Once a week, about 1 hour



424

ü Three pre-inter learners called ‘high-achievers’ here with their scores ranging from 50- 64. They will be referred to as ‘A’ in the three groups;

ü Three post-Elem learners called ‘mid-achievers’ here with their scores ranging from 35-49. They will be referred to as ‘B’ in the three groups; and

ü Three Elem learners called ‘low-achievers’ here with their scores ranging from 20-34. They will be referred to as ‘C’ in the three groups.

The following table introduces the participants of the study (with their given names). The data is based on the participants’ information obtained through their answers to two pre-study biographical/language questionnaires. The nine selected participants were assigned to three different groups with different levels of language proficiency; i.e., in each group, there were three participants: one high-achiever, one mid-achiever, and one low-achiever to set the stage for later comparisons. The design was in such a way that one group experienced the interventionist mediation, the other one the interactionist approach, and still the third one, a mixed multi-media based mediation. Instruments Before the study and to select the participants, a PET language proficiency test, listening part as well as the listening part of the Oxford Placement Test (2004) were used to assess the overall listening comprehension skill of the learners. As another instrument used in this study was CDLT (Figure 1) which stands for Computerized Dynamic Listening Test. Since one of the goals of this investigation was to compare the effect of using multimedia mediation on participants’ listening comprehension development, this software was designed in order to act as a mediator during this section of the study.

Figure 1: The software designed for the study

The software package had been designed in such a way that any PC could run it easily and could be installed properly on any computer. On the opening page of the software, some icons were designed to help the participants go through the process easier. It should be mentioned here that



425

before the study, they went through a tutorial that explained how to go through the activity and offered a sample to minimize the exploratory behavior sometimes found in the initial steps of using this kind of program (Hubbard, 2004). The first page of the software was designed in such a way that it guided the participants to move smoothly through the tasks. There were some icons whose mission was to act as mediators during the study, such as help, useful expressions, and transcript. On the ‘help’ option, the software provided the participants with a short and simple description of the software both in English and Persian and it was up to participants which description to choose and read. After reading the description, they could start the assessment. On the ‘useful expression’ option, some of the expressions used in the extracts were presented to participants and the ‘transcript’ option provided them with the complete text of the extracts. By starting the test, the first phase appeared on the screen. They were to choose ‘extract one’ and then start listening to the first extract. During the first listening, they couldn’t get help from any of the icons, since during their first non-dynamic listening, they should listen independently, recall it and record their voice after finishing the part. However, while listening to the same extract for the second time, they were allowed to use any help option as many times as they needed to and they could even replay a part if they were not able to comprehend that. Also, they could have an impeded view of the extract transcript. After that, they were required to record their voice. Afterward, they could go to the next part. It should be mentioned that the software was designed in a way that only if each participant completed the listening task, she could have gone to the next part. Moreover, if she moved to the next part, it was impossible for her to return to the previous part. After each listening, a result section was shown which showed:

1. The number of times each participants used useful expression help option; 2. The number of times each participant used transcript help option.

Materials In order to provide listening materials that would correspond to the study requirements, it was decided to use some extracts from the books taught at university level to cover the term syllabus as well. Since the participants’ language proficiency levels were different, seven different segments were selected for each group. The selected segments were based on the functions of language use like greeting, introducing oneself, accepting and refusing an invitation, and so on calibrated to the learners’ levels. The extracts were used in dependent and independent sections of the study plus the carry over sections. The texts were the same for all the participants with the same level of proficiency, i.e., all the high achievers listened to the same extracts in the three groups, for instance. Additionally, for the three participants who experienced the multi-media version of mediation, the same extracts were played electronically using the designed software. Besides, this study adopted the methodological suggestions advanced by L2 research while implementing pausal unit analysis and recall task for assessing comprehension of L2 aural texts, an example of which is provided below (Figure 2).



426

Figure 2: A sample scoring instrument based on the idea units of the text used for NDA1 session, low-level To measure comprehension, the participants were asked to recall the texts as much as they could of what they had just listened to with a special focus on the main ideas of the text.Then, the researcher identified the relationship between propositions (idea units) of the original text and the texts of participants’ recall. In this fashion, propositions from the learners’ immediate oral recalls that were closely related to propositions from the original text were counted and scored against the list of idea units designated independently by the raters before the study. The number of recalled idea units was viewed as evidence of text comprehension. Data Collection In order to compare the potential effectiveness of the the three approaches aforementioned, the methodologies advocated by Budoff, Feurestein, and a computer-assisted model were taken into consideration while elaborating the pedagogical experiment. Therefore, three different designs were proposed and conducted for this investigation to make this comparison feasible. The three groups assigned in this study were the interventionist (IV), the interactionist (IT), and the multi-media (MT). All the three groups experienced four stages in the study: pre-test, training (the IV group) / the Enrichment program (the IT and the MTgroups) , post-test, and carryover sessions. Considering the point that the students were at different levels of proficiency and the main goal of the study was to assess listening comprehension of each individual dynamically, each of the participants was assessed individually during the three main phases of the study and just the Enrichment Program was conducted in group for each of the pre-assigned groups. The Findings Throughout the study, each pre-test/post-test/carryover session in the IT and the MT groups normally involved two stages: one independent unassisted performance (IP, conducted non-

Extract 1: NDA1 D1: Hi, Marta!

S1: Are you going to the movies with us after class tonight? D2: Hi, Steve!

D3: Sure. S2: I’d love to.

D4: By the way, M1: have you met my cousin Tammy?

M2: She’s visiting us from New York for a few days. D5: Hi, Tammy, Nice to meet you.

D6: Nice to meet you, too. S3: Tammy’s just seen the latest Johnny Depp movie.

D7: Really? D8: How was it? D9: Awesome.

D10: I love Johnny Depp. ……………………………………………………………

Main IUs: 2 Supportive IUs: 3

Details: 10 Total IUs: 15



427

dynamically) and one mediated performances (MP, carried out dynamically) (the IV group only experienced independent performance). Moreover, for the two groups, NDA1 and NDA2 consisted only of the first stage. As it was explicated before, in the present study, the participants were assigned to three different groups based on two criteria, i.e., one classification was based on their language proficiency scores (e.g., high-achiever, mid-achiever, and low-achiever) and the second one was derived from the kind of mediation they were going to receive (e.g., interventionist, interactionist, and multi-media).The following will present the results of quantitative and qualitative analyses respectively. QUANTITATIVE ANALYSIS: RESULTS AND DISCUSSION Considering the learners’ language proficiency level, the overall comparison of means demonstrated that the number of idea units recalled during each of the post-test sessions were generally higher than those recalled during the first pre-test sessions. However, CA3, the final session of the study, represented an abrupt drop in the learners’ production of acceptable IUs. The comparison of the medians demonstrated that the participants as low-intermediate EFL learners were faced with a difficult listening task, i.e., oral recalls of a foreign aural language. The data obtained showed that the high-achievers were able to comprehend the texts better while the mid-achievers lagged behind the other learners in their listening development (Table 2). Examination of the SDs and ranges showed that there were some ups and downs in the variation among the learners in the three groups, specially after the enrichment program, which point to the potential benefits of DA interventions and the enrichment program offered during the study.

Table 2: descriptive statistics of the participants based on their language proficiency

Achievement Placement Score

Pre NDA1

Pre DA1

Pre CA1

Post NDA2

Post DA2

Post NDA3

Post NDA4

Post NDA5

Post NDA6

CA2 CA3

NO of IUs

27 29 25 29 20 25 21 20 26 26 27

Mean 3.00 56.67 7.33 8.00 9.00 11.67 12.00 8.00 11.00 9.00 7.00 10.00 6.00 Median 3.00 57.00 7.00 9.00 12.00 12.00 12.00 8.00 11.00 9.00 7.00 10.00 6.00 Std. Deviation

0.000 1.528 .577 2.157 5.196 2.517 2.828 1.414 1.414

Range 0 3 1 4 9 5 4 0 0 0 0 2 2

Regarding the mediation type they received, comparison of the means and medians in the three groups demonstrated that in the IV and IT groups, the mean and the median had a zigzaged trajectory while they showed a continuous increase in the MT group up to the CA sessions. What is noteworthy is that in general , the mean increase in the MT group was more significant than that of two other groups as the highest mean in the three groups can be observed in this group.



428

Table 3: Number of acceptable (IUs) recalled independently by each of the mediational groups in each assessment session

The Interventionist Group (IV)

Achievement

Placement Score

Pre NDA1

Pre NDA2

Post NDA3

Post NDA4

Post NDA5

Post NDA6

Carry over CA1

Mean 2.00 41.00 6.33 7.67 6.33 6.67 7.67 7.00 3.00 Median 2.00 44.00 6.00 8.00 6.00 5.00 7.00 7.00 3.00 Std. Deviation

1.000 17.692 .577 1.528 1.528 3.786 1.155 2.000 2.000

Range 2 35 1 3 3 7 2 4 4

The Multimedia Group (MT)

Achievement

Placement Score

Pre NDA1

Pre DA1

Pre CA1

Post NDA2

Post DA2

Carry over CA2

Carryover CA3


1.000 15.567 1.155 1.733 2.082 3.786 3.464 2.517 2.082

Range 2 31 2 4 7 6 5 4 Comparing the CA sessions, it is apparent that the lowest mean belonged to the IV group while the highest was achieved by the MT group. It may point to the possibility that the MT learners were able to benefit more in their listening development than others from the DA and enrichment program. Moreover, this group had the opportunity to experience the enriched input provided for them in form of two help options they could use during their mediated recalls which may add to the advantage. It can also point to the fact that there was a difference in the effects of the different mediation approaches used in this study. It seems that using multi-media as a new way of assessment could provide the enriched input for the MT learners promoting their level of performance (Table 3). However, it should be mentioned that the descriptive statistics appeared to point to a general improvement in listening skill of all the participants as measured by an increase in the number of IUs independently recalled from the NDA1 and DA1-IP sessions to the CA1-IP, NDA2, DA2-IP, CA2-IP and CA3-IP sessions. This improvement seems to be primarily the result of mediation and the enrichment program that was tuned to the learners’ ZPDs. Furthermore, through disaggregate analysis of the data, first the learners’ performances were analyzed first considering their level of language proficiency and then based on the mediation type they received. Pertaining to the learners’ language proficiency level, in the high achiever group, there was a progressive trend in the number of idea units each learner produced. Moreover, it was shown that

The Interactionist Group (IT)

Achievement

Placement Score

Pre NDA1

Pre DA1

Pre CA1

Post NDA2

Post DA2

Carry over CA2

Carry over CA3


1.000 15.631 1.732 2.887 4.583 1.732 2.000 1.528

Range 2 31 3 5 9 3 4 3



429

the number of IUs the MT learner in this group (i.e., A3) produced was more significant than those of her group-mates. The possible reason was that the learners in the MT group had the opportunity to experience the help options whenever needed. However, considering regression and progression, the count of the learners’ IUs in the three groups demonstrated a regression in CA1/CA3 session suggesting that even though the learners’ ability to comprehend the texts was improving, they were not able to fully control the task to the complexity of the task. The same process was observed in the mid- and low achiever groups. However, it appeared that the IV learner experienced a constant regression and progression. The possible reason was that the IV learners did not have the opportunity to enjoy mediation during the study sessions. Here again, it seems that the IUs the learners in the MT group produced increased constantly which may be due to their access to the help options as an extra advantage whenever needed. In the low achiever group, the count of the learners’ IUs in the three groups exhibited a regression in TA1/CA3 session for the IV and IT which may be due to the complexity of texts in time 3. Here again, the MT learner shows the highest number among the low-achievers. With regard to the mediation types the learners experienced, the number of IUs produced by the learners increased significantly from NDA1 to NDA2 and then to CA2 which can point to the development in the learners’ listening performance from session to session (Table 4). However, the number of IUs produced in NDA6 in the IV group and CA3-IP sessions reduced significantly comparing to CA2-IP which can be explained referring to the difficulty level of the texts as well as the conflictual nature of development that inevitably witnesses not only progression but also regression as naturally occurring characteristics of development in the ZPD (Vygotsky, 1978).

Table 4: Raw scores summary of total number of IUs produced by each participant in all sessions of the study Carryover No of IUs produced by the participants

Post-Test No of IUs produced by the participants

Enrichment/ training

Pre-Test No of IUs produced by the participants

Placement scores

Participants

CA3-IP

CA2-IP

NDA6 NDA5

NDA4

NDA3

DA2-IP

NDA2

CA1-IP

DA1-IP

NDA1

A A 7 9 11 8 A 9 3 A 7 57 IV-H: A1 A A 5 7 5 6 A 8 2 A 6 44 IV-M: B1 A A 9 7 4 5 A 6 5 A 6 22 IV-L: C1 5 9 A A A A 10 12 12 9 8 58 IT-H: A2 5 8 A A A A 7 6 7 7 5 39 IT-M: B2 3 7 A A A A 7 3 7 5 5 27 IT-L: C2 7 11 A A A A 14 14 12 10 7 55 MT-H: A1 4 6 A A A A 8 7 9 8 5 37 MT-M: B1 8 9 A A A A 8 7 8 7 5 24 MT-L: C1 32 50 21 23 20 19 54 72 65 46 54 - Total



430

Interventionist group: IV; Interactionist group: IT; Multi-media group: MT High-achiever: H; Mid-achiever: M; Low-achiever: L NDA: non-dynamic assessment DA: dynamic assessment CA: carryover session IUs: idea units IP: Independent recall A: the person was not included in the session * For the IV group, CA1 session followed all NDA sessions. In order to find out whether an interventionist, interactionist, or multi-media (MT) DA procedure could enhance the development of listening skill among EFL language learners and to see if there were any differences between the developmental processes of the three participants receiving each type of mediation, the number of IUs produced by each of the learners in the three groups were examined in detail. With regard to the learners in the IV group, listening comprehension performances of the high-achiever (A1), the mid-achiever (B1), and the low-achiever (C1) in this group were analyzed. A1’s microgenetic listening development showed a continual improvement which can be attributed to her positive response not only to the enrichment program but also to the mediation offered during the assessment session. On the other hand, the microgenetic analysis of B1’s IU production showed that her production of IUs was quite uneven, indicating an increase and decrease in her listening development. It may point to the fact that in some sessions, her ZPD was not still ripening. Also, C1 listening performance was rather insignificant. It pointed to her low motivation during the study which might have had an impact on her very moderate listening development. Thus, it can be claimed that motivation can be accounted for as an influential factor in listening comprehension.

Table 5: The summary of the number of expected and observed IUs produced by each participant in the IV group in all independent recall sessions

Total

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No Of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

participant sessions

CA1-IP NDA6 NDA5 NDA4 NDA3 NDA2 NDA1

54 3 27 7 26 9 20 11 21 8 25 9 29 7 27 A1 39 2 22 5 23 7 13 5 16 6 23 8 23 6 20 B1 37 3 25 7 11 7 15 3 9 7 22 5 15 5 25 C1

The microgenetic analysis of the listening comprehension of the learners in the IT group revealed that like A1, A2 (the high-achiever) experienced a continuous improvement in her listening performance from session to session. However, she produced more IUs than her peer A1 in the IV group and her recalls also witnessed a balanced increase of IU production. Based on her own



431

claim, it could be due to her better knowledge of vocabulary and her access to the mediator through out the study sessions, an advantage A1 could not benefit from. B2’s (the mid-achiever) developmental path was rather uneven like that of her peer microgenetic pathway in the IV group. Although B2’s listening micro-gains followed a zigzagged trajectory, she exhibited development in her IU production, which may be primarily explained by her mediated exposure to listening. An additional factor that might influence her listening improvement was her high motivation with regard to participation in this study. C2 (the low-achiever) listening comprehension was better than her groupmate B2 at mid-level. At first, Her scores indicated continuous ups and downs in the number of IUs produced. This certainly points to the uneven changes in her listening development. As in B2’s case, C2 positive attitude and enthusiasm toward her participation in the study and active engagement in the listening activities may have contributed to the micro-gains concerning her listening development. Table 6: Summary of the number of expected and observed IUs produced by each participant in the IT group in all

independent recall sessions

As it was mentioned before, the learners in the MT group had the advantage to enjoy both the mediator’s help plus the helps provided by the computer. Considering the learners’ performances in this group, A3 (the high-achiever) produced the highest count of IUs in the study exhibiting substantial development in her IU production. It can be primarily explained by her doubled mediated exposure to listening. An additional factor that might influence her listening improvement was her English proficiency level and high motivation with regard to participation in this study. B3 (the mid-achiever), whose total IUs count was the lowest in her group, even lower than C3’s, showed moderate development in her IU production. She exhibited instances of development in relation to IU production which may have been influenced by her exposure to multi-media interventions and the enrichment program as well as her positive stance concerning English language learning in general as well as her active engagement in the listening activities. What was interesting was that C3, as the low-achiever in this group, produced the highest count of IUs compared to the other two low-achievers in the IV and IT groups. Her production of IUs was continuously increasing, indicating improvement of her listening performance from session to session. The microgenetic analysis of her IU productions also demonstrated that in comparison to the other low-achievers in the study, C3’s post-test and CA results showed a greater improvement in her listening development. She even produced more IUs compared to her group-mate C2

Total

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No Of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.


CA3 CA2 DA2 NDA2 CA1 DA1 NDA1 66 6 27 9 26 10 20 12 21 12 25 9 29 8 27 A2 39 2 22 5 23 7 13 6 16 7 23 7 23 5 20 B2 42 5 25 9 11 7 15 4 9 5 22 6 15 6 25 C2



432

which may indicate a more considerable development in her ZPD. Her significant improvement can be due to different factors such as her intense interest in learning, high degree of motivation in addition to her exposure to double sided mediation.

Table 7: Summary of the Number of expected and observed IUs produced by each participant in the MT group in all

independent recall sessions Total

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No Of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.

No of IU obs.

No of IUs exp.


CA3 CA2 DA2 NDA2 CA1 DA1 NDA1 75 7 27 11 26 14 20 14 21 12 25 10 29 7 27 A3 47 4 22 6 23 8 13 7 16 9 23 8 23 5 20 B3 52 8 25 9 11 8 15 7 9 8 22 7 15 5 25 C3

Another advantage in the MT group was that they could use two help options. The three learners in this group showed a decrease in the number of times they used the help options which may point to development of their ability to comprehend and recall the texts independently (Table 8).

Table 8: Number of times MT participants used the ‘transcript’ and ‘useful expressions’ help options No. of times useful expressions option has been used

No. of times transcript option has been used

Number of IUs

Sessions

A3 (The high-achiever)

8 5 29 DA1 7 7 25 CA1 6 5 20 DA2 5 4 26 CA2 26 21 100 Total No. of times useful expressions option has been used


Number of IUs

Sessions

B3 (The mid-achiever)

11 8 23 DA1 10 6 23 CA1

7 5 13 DA2 4 6 23 CA2 31 25 82 Total No. of times useful expressions option has been used


Number of IUs

Sessions

C3 (The low-achiever)

7 10 25 DA1 8 9 22 CA1 6 4 15 DA2 7 6 11 CA2 28 29 73 Total



433

However, the analyses showed that the highest number of use belonged to C3 which may be because of her low language proficiency level. Her count demonstrated that her use followed a zigzagged trajectory indicating progression and regression in her listening development. Therefore, considering these points, it can be asserted that there was a difference in the use of help options between MT participants at different proficiency levels. The low-achiever learner used more help options than her two group-mates. It seems that all the participants, spcifically the low achievers, could benefit more from the program in the IT and MT groups which can be resulted from the materials presented to them in the enrichment program sessions. They had also the opportunity to enjoy the mediator’s assistance during the DA and CA sessions of the study which as they claim, could provide them “some hints and effective feedback”. Also, they believed that having their teacher beside them while listening made them feel “more comfortable and self-confident”, and that “ listening to the text was not difficult anymore”. In addition to, the MT group could benefit from a new way of assessment which proved to be effective in the EFL learners’ process of listening comprehension development. To see if a multi-media DA (MT) procedure could improve the development of listening skill among EFL language learners and to see if there were any differences between the developmental processes of the three participants receiving the multi-media mediation, the MT participants’ use of the two help options (transcript and the useful expressions) was analyzed. Based on the findings, the number of times A3 used the ‘transcript’ help option has increased from DA1 to CA1. However, it is followed by a slight decrease in DA2 and CA2 which may point to development in the learner listening performance. In general, the number of times she used the ‘transcript’ option was lower than those of her group-mates which may be due to her language proficiency level. B3’s count revealed that she used the ‘transcript’ option more than A3. Her use of the help option showed a continuous decrease proceeded by a slight increase in CA2 which can be explained by the difficulty of the CA2 text. Comparing to her two group-mates, C3’s count displayed the highest number of ‘transcript’ option use which may be because of her low language proficiency level. Although, it was shown that the count decreased from DA1 to DA2 pointing to her potential listening performance development. Again here, her count bears in witness to a slightly increasing number in CA2 probably stemmed from the text difficulty at CA2 session. Regarding the ‘useful expression’ option in the study, the data showed the number of times A3 and B3 used this help option declined gradually from session to session which may point to development of their ability to comprehend and recall the texts independently. However, C3’s use followed a zigzagged trajectory indicating progression and regression in her listening development. Therefore, considering these points, it can be asserted that there was a difference in the use of help options between MT participants at different proficiency levels. It seems that, in general, the help options could assist the MT learners listening comprehension development but more proficient users could benefit slightly more from the help options.



434

QUALITATIVE ANALYSIS OF MEDIATED PERFORMANCE: RESULTS AND DISCUSSION The goal of this part was to see what forms of mediation during DA-based pedagogical interventions best nurtured the development of listening skill of EFL language learners and to explore the types of responsive moves the learners exhibited to mediation during the DA-based pedagogical interventions. The arrangement of the mediator’s strategies in this study paralleled Aljaafreh and Lantolf’s (1994) regulatory scale in that it followed the principle of abstract (most implicit) to concrete (most explicit). For example, Replay was a more explicit strategy than Accepting Response or Structuring the Text since it encouraged the learners to attempt to recall a portion of a text a second time.

Figure 3: Typology of mediator‘s strategies occurred during Dynamic Assessment of L2 listening comprehension It should be specified that the strategies frequently occurred during each dynamically conducted assessment and for this reason they were selected as the major strategies since they proved to be effective in promoting learners’ listening ability. Similar to Poehner’s (2005) study, the DA-based mediation did not always follow the hierarchical order of strategies. The strategic mediation was tightly linked to the learners’ needs and the mediator had to adjust her assistance, going back and forth, depending on the specific response of the learners. In order to conduct a more thorough microgenetic analysis of the learners’ listening performance, the moves that occurred in response to mediation during the DA-MP and CA-MP sessions were further grouped into two interrelated categories: (1) a category was comprised of moves that reflect distinctive characteristics of listening development within the ZPD; and (2) the second category contained moves related to the effects of DA-based instruction observed during the assessments. The analysis suggested that, while working in the ZPD, the learners tended to produce various contrastive responsive moves. These contrastive moves can be explained by the fact that the ZPD, by definition, consists of maturing and unstable functions. The instability of maturing listening ability was examined in the study through the notions of progression and regression.

1. Accepting Response 2 Structuring the text 3. Replay of a passage; Replay of a segment (from a passage); Replay of a detail (from a segment) 4. Asking the Words 5. identifying specifc site of problem 6. Metalinguistic Clues 7. Offering a Choice 8. Translation 9. Providing a Correct Pattern 10. Providing an Explicit Explanation



435

The rationale for this examination was in line with the SCT genetic approach which reflects Vygotsky’s double-sided view of development as an evolutionary as well as revolutionary process (Vygotsky 1978; 1997a). To track developmental changes of L2 listening ability over time, the study microgenetically captured the frequency of mediational strategies and responsive moves at three specific stages of DA-based interventions, i.e., DA1, DA2, and CA3. Comparisons of the selected sessions displayed the microgenetic growth of the learners’ZPD and developmental changes that occurred at three different points in time during the study (Table 9).

Table 9: Mediator’s moves within the learners’ZPD (during DA1-MP, DA2-MP and CA3-MP)

Analysis of the findings showed that from among the strategies delivered by the mediator, ‘Replay’, specifically ‘Replay of a segment’ or ‘Focused replay’, was the most frequent strategy used by the mediator. However, the fluctuation in the number of times this strategy was offered could confirm the results obtained through quantitative analysis by reflecting learners’ regressive and progressive movements, which, in turn, indicated the growth of the learners’ listening comprehension ability. Furthermore, the observed decrease in the need to translate and to provide correct pattern might also indicate progressive changes in the learners’ ZPD, leading to development (Table 9).

Participant

A2

A3

B2

B3

C2

C3

Category DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3

Accepting response

25

16

18

23

15

20

17

12

19

21

13

11

15

9

11

14

10

11

Replay of a paragraph

7 5 5 10 5 6 11 7 6 12 7 2 14 7 4 11 5 2

Replay of a seg./det.

14 8 18 23 9 24 28 11 30 35 24 38 30 25 31 31 23 31

Structuring the text

1 1 0 1 2 1 1 2 2 1 1 3 2 3 1 2 2 1

Asking the words

2 1 1 2 0 2 3 1 1 3 2 0 3 1 2 2 1 3

Identifying problem area

2 3 5 4 2 5 4 1 6 6 3 4 7 4 3 9 1 3

Metalinguistic clues

3 1 2 4 3 1 7 2 7 8 0 9 9 1 2 9 6 0

Offering a choice

1 0 3 2 4 1 3 4 2 5 2 1 4 0 3 5 1 3

Translation 1 2 5 3 1 2 8 1 5 13 1 3 11 4 8 11 4 6

Providing correct pattern

8

5

5

10

2

3

8

2

3

8

6

5

9

4

7

11

5

5

Providing explanation

2 2 1 4 1 2 6 4 2 7 5 4 10 3 4 10 2 5



436

With regard to total number of mediational strategies used at three different points in the study, a significant decrease was observed in their use at time 2 as compared to time 1 and a slight decrease at time 3 as compared to time 1. This tendency in strategies use can be explained by the varying complexity of the texts used in the study: the texts selected for DA1 and DA2 were selected from among the basic functions of language identical in terms of format (e.g., introducing others, giving and receiving information, etc.), whereas the texts selected for CA3 were different in a more substantial way, i.e., in genre (monologue) and in topic (e.g., instruction, report). Be it as it may, the frequency of total strategies used by the mediator clearly showed the learners’ movement towards self- regulation concerning their ability to understand spoken English.

Table 10: Learners’ moves within the ZPD (during DA1-MP, DA2-MP and CA3-MP)

Participant

A2

A3

B2

B3

C2

C3

Category DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3 DA1 DA2 CA3

Responsive

1

2

2

0

1

4

3

4

7

4

2

6

9

2

8

7

6

8

Unresponsive 0

3

6

2

0

1

3

2

3

3

2

4

8

2

11

8

3

7

Providing positive response

13

8

15

13

5

17

18

12

16

14

11

21

24

15

19

25

14

23

Providing negative response

10

3

9

10

5

7

14

3

8

17

4

7

26

17

13

27

14

10

Making a right choice

2

1

1

2

3

0

0

0

2

1

2

2

1

0

1

1

4

2

Making a wrong choice

1

0

3

1

1

2

2

3

1

1

3

1

8

1

3

6

2

2

Decipher a pattern

14

9

19

16

10

21

16

6

18

14

9

17

14

12

16

10

12

20

Doesn’t decipher a pattern

7

1

3

8

2

7

14

7

5

14

3

4

15

10

7

12

8

7

Overcomes a problem

10

3

11

8

5

11

14

5

4

10

12

6

7

4

8

8

5

10

Doesn’t overcome a problem

4

4

3

4

2

3

7

2

4

7

1

3

5

9

3

4

6

5

Total responsive moves

62

34

72

64

34

73

91

44

68

85

49

71

117

72

89

108

73

94

Total progressive move

40

23

48

39

24

44

51

27

47

43

36

52

55

33

52

51

41

63

Total regressive moves

22

11

24

25

10

29

40

17

21

42

13

19

62

39

37

57

32

31



437

Considering the learners’ moves, the moves ‘Decipher a pattern correctly’ and ‘overcomes a problem’ were considered here to be important indicators of listening development over time since they demonstrated the learners’ emerging ability to function independently without reliance on the mediator’s assistance. The slight increase in frequency of this move at time 3 (CA3) provided clear signs of the learners’ improved ability to understand aural spoken English. That is, the counts of these two moves suggested that even though the learners still required mediation and were not able to fully control the task, their ability to comprehend spoken texts was improving since at time 3 they were asked to recall a more complex text as compared to the texts at time 1 and 2. In addition, the counts of the responsive move ‘Does not decipher a pattern’ and ‘doesnot overcome a problem’ tended to decrease over time. This responsive move is the mirror image of ‘Decipher a pattern correctly’ and its decrease is regarded here as an indication of development of listening comprehension skill. With regard to regression and progression, the counts of learners’ responsive moves revealed that generally the learners tended to produce more progressive than regressive moves with the number of progressive moves shows an impressive increasing impressively from session to session. From a Vygotskyan perspective, this tendency clearly points to the growth of the learners’ listening ability in the ZPD. CONCLUSION AND PEDAGOGICAL IMPLICATIONS The investigation of mediated listening performance allowed for diagnosing several factors that impede L2 listening comprehension at the three levels of language proficiency in question. As evident from the qualitative analysis, the mediational phases of the DA and CA sessions, as opposed to non-mediated phases and NDA sessions, offered a fuller picture of the difficulties experienced by the participants while listening to the texts. The problem areas that impeded successful listening performance resided in all aspects of language learning, i.e., lack of effective listening strategies, functional knowledge of language use, phonology, lexis, grammar, and some other factors. The diagnostic analysis revealed that grammatical and phonological problems present a noticeable challenge for all the learners at these levels. The analysis demonstrated that for these learners L2 phonology constituted one of the biggest challenge in text comprehension since poorly developed L2 phonology did not allow them to recognize previously acquired words and compelled them to confuse L2 sounds. Moreover, this study explored whether strategy instruction might have any effect on the learners listening comprehension improvement in the foreign language listening classrooms. It seems that the results speak in favor of such a role. That is to say, strategy training is intended to raise the students’ awareness of metacognitive strategies, through which the learner is more able to select strategies appropriate to a particular problem, rather than engage in the activity for its own sake. To make it simple, students should not be only taught what to learn, but also how to learn. To conclude, it should be noted that the traditional idea of only exposing EFL students to listening texts in listening classes should be challenged by an approach in which strategies can effectively and successfully be embedded in the listening course by means of strategy training



438

program. The knowledge of language that the students need is that of vocabulary and grammatical structures. At the same time, students should be trained about how to effectively use listening strategies. The more strategies they know, the more beneficial it will be for them. Besides, any approach to listening instruction requires teachers to have relevant knowledge of, and know how to implement, appropriate techniques, not just listening strategy instruction. I would suggest that most teachers have the capacity to develop the knowledge and ability, either by themselves or through ‘In Service English Teacher Training’ sessions, to include elements of strategy instruction in their listening lessons within allocated course times. Limitations of the study and suggestions for further research There were several limitations to the study that need to be acknowledged and addressed. The limitations described below may also be regarded as orientation for potential future L2 DA research and/or L2 listening. The first limitation concerns the length of the study. Despite the longitudinal nature of the present study, the development of the learners’ listening ability was tracked only within nine-month period of time. Accordingly, further investigation that would track development of listening ability over longer periods would offer more insights and refine our understanding of L2 listening comprehension processes. The second limitation had to do with the extent to which the findings can be generalized to EFL university learners. The number of learners participating in this study was too limited for broad generalizations. Also, the participants in this study were female EFL learners. Consequently, more investigation can be done using both male and female learners. Moreover, the present study was conducted among three different levels of language proficiency; therefore, further empirical evaluations are needed to replicate the findings in different contexts and possibly at other levels of language proficiency. The future investigations can potentially replicate the design of the present study but should take into account of the third limitation discussed in the next paragraph. The third limitation relates to the carry-over tasks. The texts used in the CA2 and specifically in the CA3 sessions caused severe comprehension difficulties for the learners when recalled independently. In order to calibrate carry-over assessments to the leaners’ microgenetic gains, it would perhaps be more appropriate to design the tasks in terms of text selection after the first NDA and DA sessions and/or after the enrichment program rather than a priori, as was done in the present study. Finally, more rigorous diagnostic qualitative analyses of L2 listening processes via DA should be performed. This dissertation has provided diagnostic insights only into the processes related to listening comprehension of EFL university students. To inform listening instruction, it would be necessary to conduct a range of empirical studies in order to reveal problem areas that cause comprehension difficulties of L2 learners studying various languages and at different proficiency levels. The focus of these studies should be on the degree to which each of the components of listening comprehension has an influence. With regard to listening research, one factor that could be considered in future studies is the role of non-verbal behavior, including gesture, facial



439

expressions, body posture, etc. This factor was left unexplored in the present study. Thus, future studies offering empirical insights into the role that non-verbal behavior plays in L2 listening would also enhance our understanding of the processes involved in L2 aural text comprehension. REFERENCES Ableeva, R. (2007). Assessing listening for development. In R. Alanen & S. Poyhonen (Eds.),

Language in action: Vygotsky and Leontievian legacy today (pp. 352-379). Cambridge Scholars Publishing.

Ableeva, R. (2008). The effects of dynamic assessment on L2 listening comprehension. In J.P. Lantolf & M. Poehner (Eds.), Socio-cultural theory and the teaching of second languages (pp. 57-86). London: Equinox Press.

Antón, M. (2003). Dynamic assessment of advanced foreign language learners. Paper presented at the American Association of Applied Linguistics, Washington, D.C., March, 2003.

Antón, M. (2009). Dynamic assessment of advanced second language learners. Foreign Language Annals, 42 (3), 576-598.

Borras, I., & Lafayette, R. (1994). Effects of multimedia courseware subtitling on the speaking performance of college students of French. The Modern Language Journal, 78 (1), 61-75.

Brett, P. (1996). Using multimedia: An investigation of learners’ attitudes. Computer Assisted Language Learning, 9 (2-3), 191-212.

Brett, P. (1997). A comparative study of the effects of the use of multimedia on listening comprehension. System, 25 (1), 39-53.

Chapelle, C. A. (1998). Multimedia CALL: Lessons to be learned from research on instructed SLA. Language Learning and Technology, 2 (1), 22-34.

Chapelle, C. A. (2003).English language learning and technology: Lectures on applied linguistics in the age of information and communication technology. Amsterdam: John Benjamins.

Chun, D. M., & Plass, J. L. (1997). Research on text comprehension in multimedia environments. Language Learning & Technology, 1 (1), 1-35.

Feuerstein, R., Feuerstein, R. S., Falik L.H., & Rand, Y. (2002). The dynamic assessment of cognitive modifiability. The ICELP Press: Jerusalem

Haywood, H. C., & Lidz, C.S. (2007).Dynamic assessment in practice: Clinical and educational applications. Cambridge: Cambridge University Press.

Lantolf, J. P., & Poehner, M. E. (2004). Dynamic assessment of L2 development: Bringing the past into the future. Journal of Applied Linguistics, 1 (2), 49-72.

Lantolf, J. P., & Poehner, M. (2008). Dynamic assessment. In N. Hornberger & E. Shohamy (Ed.), The encyclopedia of language and education, (vol. 7) (pp. 273-285): Language Testing and Assessment. Cambridge: Cambridge University Press.

Long, M. H. (1996). The role of linguistic environment in second language acquisition. In W. C. Ritchie, & T. K. Bhatia, (Eds.), Handbook of second language acquisition (pp. 413-468). San Diego: Academic Press.

Pica, T. (1994). Research on negotiation: What does it reveal about second-language learning conditions, processes, and outcomes? Language Learning, 44(3), 493-527.



440

Plass, J.L., Chun, D. M., Mayer, R. E., & Leutner, D. (1998). Supporting visual and verbal learning preferences in a second language multimedia learning environment. Journal of Educational Psychology, 90 (1), 25-36.

Poehner, M. E. (2005). Dynamic assessment of oral proficiency among advanced L2 learners of French. Unpublished dissertation. Pennsylvania State University, University Park.

Poehner, M.E. (2008). Dynamic assessment: A Vygotskian approach to understanding and promoting second language development. Berlin: Springer Publishing.

Poehner, M. E., & Lantolf, J.P. (2005). Dynamic assessment in the language classroom. Language Teaching Research, 9, 233-265.

Pujolà, J. T. (2002). Calling for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL, 14 (2), 235-262.

Sternberg, R. J., & Grigorenko, E. L. (2002). Dynamic testing. The nature and measurement of learning potential. Cambridge: Cambridge University Press.

Vygotsky, L.S. (1986). Thought and language. Cambridge, MA: MIT Press. Vygotsky, L.S. (1987). Thinking and speech. In R. W. Rieber & A. S. Carton (Eds.).The

collected works of L. S. Vygotsky (Vol. 1): Problems of general psychology. New York: Plenum.

Vygotsky, L. S. (1998). The problem of age. In R. W. Rieber (Ed.), The collected works of L. S. Vygotsky (Vol. 5): Child psychology. New York: Plenum.

international journal of language learning and applied ...ijllalw.org/finalversion5134.pdfbecome as...

Documents