anybody listening? the role of listening in integrated listening-to … · 2015-12-02 · what...
TRANSCRIPT
Anybody listening? The role of listening in integrated listening-to-write and listening-to-speak tasks
Anchana Rukthong & Tineke Brunfaut LTF 2015, Oxford
Please cite as:
Rukthong, A. & Brunfaut, T. (2015). Anybody listening? The role of listening in integrated listening-to-write and listening-to- speak tasks. Paper presented at LTF 2015. Oxford, UK, November, 20-22.
Copyright © Rukthong, A. & Brunfaut, T., 2015
What?
Integrated-skills assessment
Information processing & transferring
Input text(s)
Listening Reading Speaking Writing
Task production
Why?
• Strengths
‒ More authentic test tasks (Asencion, 2004; Cumming et al., 2004)
‒ Positive washback (Weigle, 2004)
‒ Fairness (Cumming et al., 2004)
‒ Greater inter-rater reliability (Weir, 2004)
• Weaknesses
‒ Unclear test construct (Weir, 1990)
‒ Difficulty in score interpretations (Lewkowicz, 1997)
Integrated-skills assessment
Why research integrated-listening tasks?
1. Research focus re integrated-skills tasks primarily on productive skills (E.g., Brown et al., 2005; Frost et al., 2011), not receptive skills
2. No consensus on role of comprehension in integrated-skills tasks (E.g., Gebril, 2004 vs Plakans, 2008)
3. Comparatively limited insights in listening (Field, 2013;
Lynch, 2009; Rost, 2011)
Integrated-skills assessment
Listening comprehension processing
Cognitive processes
Adapted from Anderson (1985), Rost (2011), and Field (2013)
Adapted from Goh and Vandergrift (2012)
Comprehension processing
Text understanding
Cognitive strategies
Metacognitive strategies
Input
Adapted from Rost (2011), Goh and Vandergrift (2012)
Listening comprehension processing
Acoustic-phonetic processing Pragmatic processing
Parsing
Word recognition Semantic processing (global)
Semantic processing (local)
Cognitive processes
Reconstruction
Translation Prediction
Elaboration
Inferencing
Fixation Comprehension evaluation
Real-time assessment of input
Comprehension monitoring
Directed attention
Selective attention
Preparing for listening
Metacognitive strategies
Cognitive strategies
What listening abilities are assessed by integrated listening-to-summarize tasks?
a) What listening processes and strategies do ESL test-takers engage in while performing integrated listening-to-summarize tasks?
b) Are there any differences in the processes and strategies when compared between task modalities and performance levels?
Research questions
Materials
‒ Integrated listening-to-summarize tasks
‒ 4 tasks adapted from the PTE Academic
‒ 2 versions per task
Methodology
Listening
Listening-to-speak task
Listening-to-write task
Speaking or Writing
Oral/written summary
Participants
• 12 Thai-L1 students from a UK university
• 6 female, 6 male
• Age range: 23-40y (M= 29)
• English proficiency (IELTS): ‒ Overall: 6.0-8.5
‒ Listening: 5.5-7.5
‒ Reading: 5.5-8.5
‒ Writing: 5.5-8.0
‒ Speaking: 5.5-8.5
Methodology
Procedures
Methodology
Task 1
Background Questionnaire
Task 2
Task 3
Task 4
Stimulated recall
Stimulated recall
Stimulated recall
Stimulated recall
Data collection
Methodology
Stimulated recalls
Participants’ notes
Written summary content
…his father told him that….
Oral summary content
Analyses
• Summaries marked by two experienced PTE Academic raters
• Written summary: content, grammar and vocabulary
• Oral summary: content, pronunciation, fluency
• Stimulated-recall data + participants’ notes + summary content
• 2 coders
• Comprehension processing model
• Agreement (Kappa): 0.77- 0.88
Methodology
Findings (RQ1)
0
2
4
6
8
10
12
14
Acoustic-phoneticprocessing
Word recognition Parsing Semanticprocessing (local)
Semanticprocessing
(global)
Pragmaticprocessing
Par
tici
pan
ts
Cognitive Processes
Listening A
Listening B
Listening C
Listening D
0
2
4
6
8
10
12
14
Acoustic-phoneticprocessing
Word recognition Parsing Semanticprocessing (local)
Semanticprocessing
(global)
Pragmaticprocessing
Par
tici
pan
ts
Cognitive Processes
Listening A
Listening B
Listening C
Listening D
Lower –level processes Higher –level processes
Findings
0
2
4
6
8
10
12
14
Prediction Fixation Inferencing Elaboration Reconstruction
Par
tici
pan
ts
Cognitive strategies
Listening A
Listening B
Listening C
Listening D
Findings
0
2
4
6
8
10
12
Preparing forlistening
Selectiveattention
Directedattention
Comprehensionmonitoring
Real-timeassessment of
input
Comprehensionevaluation
Par
tici
pan
ts
Metacognitive Strategies
Listening A
Listening B
Listening C
Listening D
Interaction between processes and strategies
Comprehension monitoring
Fixation
Directed attention
Listening input
Acoustic-phonetic processing
Word recognition
Semantic processing (local)
Semantic processing (global)
Pragmatic processing
Parsing +
-
Low
er-l
ever
Interaction between processes and strategies
Inferencing
Elaboration
Listening input
Acoustic-phonetic processing
Word recognition
Semantic processing (local)
Semantic processing (global)
Pragmatic processing
Parsing
Comprehension monitoring
H
igh
er-l
ever
Listening-to-speak vs. listening-to-write:
Findings (RQ2)
Cognitive processing
Listening-to-speak Listening-to-write
Aco
ust
ic-p
ho
ne
tic
Wo
rd r
eco
gnit
ion
Par
sin
g
Sem
anti
c (l
oca
l)
Sem
anti
c (g
lob
al)
Pra
gmat
ic
Aco
ust
ic-p
ho
ne
tic
Wo
rd r
eco
gnit
ion
Par
sin
g
Sem
anti
c (l
oca
l)
Sem
anti
c (g
lob
al)
Pra
gmat
ic
Hig
h
sco
rer Listening A
Listening B
Listening C
Ave
rage
sc
ore
r
Listening A
Listening B
Listening C
Listening D
Low
sc
or
er
Listening A
Listening D
er
Directed attention
Co
mp
rehen
sion
Mo
nito
ring
Acoustic-phonetic processing
Parsing
Word recognition
Semantic processing (local)
Semantic processing (global)*
Pragmatic processing*
Infe
ren
cin
g
Elab
ora
tio
n
Co
mp
reh
ensi
on
Mo
nit
ori
ng
Fixa
tio
n
Rea
l-ti
me
asse
ssm
ent
of
inp
ut*
Inferen
cing
Elabo
ration
Directed attention
Fixation
Acoustic-phonetic processing + Word
recognition + Parsing
Pre-listening preparation
Prediction Prediction Selective attention
Listening Input
Low-scorer High-scorer
Task output
Comprehension evaluation
Reconstruction
Tow
ard
th
e en
d
At
the
begin
nin
g
Wh
ile
list
enin
g
Selective attention
• Complex interaction of comprehension processes and strategies during integrated-listening task performance
• Common processes and strategies shared between listeners, but with different levels of success
• ‘Successful listeners’:
‒ higher-level processes,
‒ real-time assessment of input
‒ more automatic and accurate lower-level processes,
‒ more accurate inferences,
‒ more effective comprehension monitoring
Conclusion
• Listening abilities play an important role in integrated-listening task performance
Conclusion
Listening should be recognised to be part of the construct tested in listening-to-speak and listening-to-write tasks
Anybody listened?
Anybody questions?
Thank you!
Anchana Rukthong & Tineke Brunfaut
We would like to thank Pearson for their help with materials