integrating icall into synchronous cmc can intelligent computer-generated feedback be provided in...

Integrating ICALL into synchronous CMC

Can intelligent computer-generated feedback be provided in learner-to-learner interactions?

Markus Dickinson, Soojeong Eom, Natalia Jacobsen, Yunkyoung Kang, Chong Min Lee, Ken Petersen, & Rebecca Sachs

iicall2 conference; December 8, 2007; Waterloo

Intelligent CALL (ICALL)

ICALL provides many potential means of facilitating L2 development Generation of detailed information regarding sources of

learner errors

Precise feedback on learner errors

Can be tailored to learners’ common mistakes, activity goals, proficiency levels, abilities, etc.

Fosters awareness of relevant language forms

Tracking of improvement across exercises, using learner models

(cf. Amaral & Meurers 2006, Heift & Schulze 2007)

Intelligent CALL: Limitations However, there is a tension between

Contextualized language use is increasingly emphasized, and with increasing success (e.g., Amaral et al. 2006; Amaral & Meurers 2006)…

But actual communicative interaction is relatively unexplored (though see Petersen 2006)

In order to manage computational complexity: Exercises are often restricted to the sentence level Activities often do not simulate true communication

The ability of an ICALL system to provide

meaningful, accurate feedback

The flexibility an ICALL system allows for in terms of meaningful, communicative

interaction

Synchronous CMC Synchronous computer-mediated communication (CMC) between

L2 learners can provide a beneficial context for L2 development

Focus on meaningful communication Activities can approximate ‘target tasks’ relevant to real-life

communicative situations, with learners functioning as language users (Ellis 2003)

Possibly optimal psycholinguistic environment for learners to negotiate for meaning and make form-meaning connections (Blake 2000, Doughty & Long 2003)

Less pressing time constraints and less ephemeral language (e.g., through possibility of re-reading) in the written modality Reduced processing demands, which may serve as an equalizer for

learners with lower working memory capacity (Payne & Whitney 2002) Attention to form and monitoring of language production (Chapelle

2003, Pellettieri 2000) Use of more complex language (Warschauer 1996)

Synchronous CMC: Limitations Concerns regarding the quality of learner-learner interactions with

respect to potential benefits for learning

The blind leading the blind? Without feedback from a trusted authority, learners might reinforce

each other’s errors (Kern 1995) and/or simply not have the resources necessary for correcting each other (Blake 2000)

In negotiating for meaning, learners may naturally tend to focus on lexis without attending to morphosyntax (Blake 2000)

Learners might not be concerned with grammatical accuracy (Kern 1995), in which case available planning time is a moot point

Teachers may need to set explicit expectations for grammatically correct language, while simultaneously trying to balance this with a primary focus on meaningful communication (Lee 2001, Pellettieri 2000)

Combining the benefits Can ICALL and synchronous CMC be integrated in

a way that exploits the unique benefits of each while avoiding their limitations?

Intelligent computer-generated feedback in synchronous, task-based, computer-mediated

learner-learner interactions

A balancing act We want learners’ interactions to be as free as

possible, promoting authentic communication

At the same time, we are restricted by limitations of ICALL processing and must constrain the situation somehow (cf. discussion in Amaral et al. 2006)

Guided picture-based information-gap tasks can constrain the vocabulary and domain in ways that: reduce many of the complexities involved in generating

feedback (computationally speaking) still allow for communicative interaction between learners

(interactional authenticity if not real-world authenticity)

Participants English-speaking university students in first-year

Korean classes

Can (or should) beginning learners use CMC? Important for learners to develop the ability to use language

to communicate meaning as opposed to simply ‘displaying’ language or manipulating L2 forms systematically without attending to meaning

However, unconstrained communicative tasks might be stressful (or a long shot) for beginning learners

We hope to design our tasks in such a way that participants will be comfortable communicating meaningfully in the L2 With appropriate guidance Within a constrained and familiar environment

A communicative task we’re considering: Spot the differences Each participant sees one version of a house and must exchange

information in the L2 in order to find similarities and differences between the two versions

Not so authentic in terms of real-world relevance, but ‘interactionally authentic’ – the sort of task often used in interaction research to target specific areas of language and promote negotiation and L2 learning

To increase motivation, set up as a competition between pairs of participants Record (in a provided chart) the activities and locations of all

characters in partner’s house Indicate whether each of these represents a similarity or a

difference between the pictures Compare scores at the end to those of other participant pairs

Spot-the-differences task (picture 1)

Spot-the-differences task (picture 2)

Target of feedback: Korean particles In Korean (as in Japanese, cf. Nagata 1995),

postpositional particles are used to indicate grammatical functions and thematic roles (e.g., who is doing what to whom).

They must be used even in simple sentences. Particles are taught from the beginning of L2 Korean

study, but: The system is quite complex and difficult to master for adult

learners of Korean Particle errors account for a substantial proportion of the

mistakes made by beginning learners (Ko et al. 2004) Errors persist even at advanced levels

Difficulties with Korean particles No one-to-one correspondence between Korean

locative particles and English prepositions

Korean locative particles mark distinctions not made in English

에 (‘e’)

에게 (‘ege’)

께 (‘kke’)

한테 (‘hante’)

English ‘to’

에 (‘e’)

에서 (‘eseo’)

을 / 를 (‘eul/reul’)*

English ‘in’

Location of a static object

Location of a dynamic activity

(*also an object marker)

Location an action moves through

Examples of targetlike and non-targetlike Korean particle use What is in the kitchen?

주방 - 에 뭐 - 가 있어요 ? 주방 - 에 뭐 -* 를 있어요 ?kitchen-LOC what-SUBJ is kitchen-LOC what-OBJ is

Father is grilling meat. 아버지 - 가 고기 - 를 구어요 . 아버지 - 가 고기 -* 가 굽고

있다 .father-SUBJ meat-OBJ grill father-SUBJ meat-SUBJ is grilling

A cat is in the living room. 고양이 - 가 거실 - 에 있어요 . 고양이 - 가 거실 -* 에서 있다 .

cat-SUBJ living room-LOC is cat-SUBJ living room-LOC is

(TL examples are on the left with particles in green; non-TL are on the right with particles in red)

SS

S

D

S D

S D

S

D

S D

S D

D D

[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해

요 ? [ParticipantB]: …

CHECK SEND

오빠가 소파를 책을 읽어요 .

Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…

In your sentence, 소파 is marked with the particle 를 , which

suggests that 소파 is an object. Instead, you

need the particle 에서 attached to 소파 in order to indicate that

소파 is the location of a dynamic activity.

울었어요 소파

에서 가 을

CHARACTER LOCATION ACTIVITY

Mother

Father

Grandma 주방 읽다…



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

TASK PICTURE:

Partners have slightly different versions and must communicate to find differences. They can scroll over the picture to enlarge it.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

WORD & PARTICLE BANKS:

To create a sentence, participants click on

words and particles…



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

…which then appear in the

sentence drafting area.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D DIf they want help with

Korean particle usage, they can

request feedback on their sentences

before entering them into the conversation.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

FEEDBACK AREA: Here, participants receive metalinguistic feedback with advice on particle usage.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

When they are ready, they click SEND to

enter their utterance into the conversation.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

CHAT WINDOW: They can scroll up and down to review the conversation so far.



CHECK SEND


Word Bank Particles

에 침대

의자

책

화장실

거실

읽었어요

는 이

…





울었어요 소파

에서 가 을


Mother

Father


S

S

D

S D

S D

S

S

D

S D

S D

D D

GAME RECORD: When participants find similarities or differences, they drag the relevant words for locations and activities here to record information about their partners’ pictures, then click on ‘S’ or ‘D’ to indicate whether the pictures match in those respects or not.

The importance of instructions Piloting so far (with native speakers of Korean in

face-to-face interactions) has suggested that carefully worded instructions will likely be needed to: Encourage participants to use complete sentences

Emphasize that the characters in the pictures are members of the same family (so that terms such as ‘mother’, ‘brother’, etc., can be used naturally)

Clue participants in to the fact that differences will involve the locations and activities of family members, as opposed to characteristics of furniture (for example)

From activity to processing How can we process CMC as input? Built-in constraints

Intensive feedback is provided on one particular error type Learners’ construction of sentences is guided by…

The nature of the picture-based task (constrains vocabulary) Instructions and the game record chart Word and particle banks, which limit the types of argument

structure by limiting the verbs that can be used, and which moreover…

+ May be necessary for beginning learners who can’t type in Korean

+ May serve as a scaffold for using receptive vocab in conversation

Upshot: Processing can focus just on detecting particle errors in a known domain.

Intensive feedback Some have argued that intensive feedback may be more effective

in certain contexts than wide-ranging incidental feedback on a variety of error types (e.g., Lyster 1998, Nicholas, Lightbown, & Spada 2001) Intensive feedback = focusing consistently on one pre-selected

error type Meta-analyses have not found significant differences, but too few

primary studies to tell for sure; in any case, both produce large effect sizes (e.g., Mackey & Goo 2007, Russell & Spada 2006)

In our study, we will inform learners that they will only be receiving feedback on particles Important for meaning (i.e., communicating who is doing what to

whom) in Korean Will hopefully prevent learners from mistaking non-feedback for

correctness Leaves open the possibility of providing other feedback, if needed

Meaningful communication? A spot-the-differences task provides a possible forum for

learners to use an L2 in purposeful, communicative ways, but…

How much will the participants focus on meaningful communication if it’s clear that the feedback is focusing exclusively on Korean particles?

What else can we do to encourage a balance between focusing on intended meanings and focusing on the forms required to express those meanings accurately?

How free will the participants be to express a creative range of ideas in the context of this task?

What can we do to make the word bank sufficiently rich for their communicative purposes?

How fluid will communication feel if participants have to take the time to search for each word they want to use?

What can we do to make the words in the bank as easy to access as possible?

Future directions Pilot the game with L2 learners

Get a clearer sense of what to expect in learner input Test how the word bank actually plays out

Develop the system Activity, error diagnosis, and feedback modules Processing techniques

Make the tasks more complex, meaningful, and relevant to real-life communicative situations Create realistic rationales for having to find differences (e.g., a

detective and witness discussing a crime scene as it appears just before and after a crime)

Explore other sorts of picture-based information-gap tasks Remove or expand the word bank

This would require additional constraints, however, to keep processing feasible

Develop activities to target more constructions

Opportunity to test techniques A combination of techniques will ultimately be used. Linguistic processing will be kept separate from error

diagnosis and feedback generation.

We can use this setting to experiment with different processing technologies

1. More traditional, anticipation-based regular expressions

2. Basic linguistic abstraction of POS tags and chunks to provide annotation of input (cf. Amaral & Meurers 2006)

3. Experimental ill-formed input checking Use mismatches in 2 different parsers to detect particle errors

(cf. Metcalf & Boyd 2006) One parser captures particle usage patterns from real language One parser captures general argument structure patterns between

words, irrespective of particles

Opportunity to test questions of SLA theory and language pedagogy We will be able to explore learners’ uses of various

types of feedback in CMC, using pre-test / treatment / post-test designs to assess L2 development Intensive vs. non-intensive feedback? Metalinguistic information vs. prompts vs. recasts? (etc.) Content-oriented feedback?

Eventually, we can use such a set-up to investigate ways of optimizing feedback: For different areas of language For learners of different proficiency levels For learners with different aptitude profiles

Using and integrating this system with the Korean language curriculum at Georgetown

Questions or comments?

Please email: Markus Dickinson [email protected] Soojeong Eom [email protected] Natalia Jacobsen [email protected] Yunkyoung Kang [email protected] Chong Min Lee [email protected] Ken Petersen [email protected] Rebecca Sachs [email protected]

References Amaral, L., & Meurers, D. (2006). Where does ICALL fit into foreign

language teaching. CALICO 2006. University of Hawai’i. Amaral, L., Metcalf , V., & Meurers, D. (2006). Language awareness

through re-use of NLP technology. Pre-conference Workshop on NLP in CALL – Computational and Linguistic Challenges. CALICO 2006. University of Hawai’i.

Blake, R. (2000). Computer-mediated communication: A window on L2 Spanish interlanguage. Language Learning & Technology, 4(1), 120-136.

Chapelle, C. (2003). English language learning and technology. Philadelphia: John Benjamins.

Doughty, C.J., & Long, M.H. (2003). Optimal psycholinguistic environments for distance foreign language learning. Language Learning & Technology, 7(3), 50-80.

Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.

Heift, T., & Schulze, M. (2007). Errors and intelligence in computer-assisted language learning: Parsers and pedagogues. Routledge.

References Kern, R. G. (1995). Restructuring classroom interaction with networked

computers: Effects on quantity and characteristics of language production. The Modern Language Journal, 79, 457-473.

Ko, S., Kim, M., Kim, J., Seo, S., Chung, H., & Han, S. (2004). An analysis of Korean learner corpora and errors. Hanguk Publishing Co.

Lee, L. (2001). Online interaction: Negotiation of meaning and strategies used among learners of Spanish. ReCALL Journal, 13(2), 232-244.

Lyster, R. (1998). Form in immersion classroom discourse: In or out of focus? Canadian Journal of Applied Linguistics, 1, 53-82.

Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research synthesis. In A. Mackey (Ed.), Conversational interaction and second language acquisition: A series of empirical studies (pp. 407-452). Oxford: Oxford University Press.

Metcalf, V., & Boyd, A. (2006, December). Head-lexicalized PCFGs for verb subcategorization error diagnosis in ICALL. Workshop on Interfaces of Intelligent Computer-Assisted Language Learning. The Ohio State University.

References Nagata, N. (1995). An effective application of natural language processing in

second language instruction. CALICO Journal, 13(1), 47-67. Nicholas, H., Lightbown, P. M., & Spada, N. (2001). Recasts as feedback to

language learners. Language Learning, 51, 719-758. Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through

synchronous CMC: Output, working memory, and interlanguage development. CALICO Journal, 20(1), 7-32.

Pellettieri, J. (2000). Negotiation in cyberspace: The role of chatting in the development of grammatical competence in the virtual foreign language classroom. In M. Warschauer & R. Kern (eds.), Network-based language teaching: Concepts and practice, 59-86. Cambridge: CUP.

Petersen, K. (2006, December). Measuring L2 development in an ICALL context. Workshop on Interfaces of Intelligent Computer-Assisted Language Learning. The Ohio State University.

Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching. Amsterdam: John Benjamins.

Warschauer, M. (1996). Comparing face-to-face and electronic discussion in the second language classroom. CALICO Journal, 13(2), 7-26.

integrating icall into synchronous cmc can intelligent computer-generated feedback be provided in...

Documents

l2 learners

learner errorscan

quality of learner

learner modelscf

ephemeral language

correct language

learners common mistakes

contextualized language