practicing russian listening comprehension skills in ...€¦ · programming • applied vr/ar...
TRANSCRIPT
![Page 1: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/1.jpg)
Practicing Russian Listening
Comprehension Skills in Virtual
Reality
Ewa Golonka, Medha Tare, Jared Linck, Sunhee Kim
PROPRIETARY INFORMATION
© 2018 University of Maryland. All rights reserved.
![Page 2: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/2.jpg)
Virtual Reality at UMD
2
Home of the AUGMENTARIUM virtual and augmented reality laboratory and
the OCULUS-CEO funded Brendan Iribe Center
![Page 3: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/3.jpg)
VR for language training at UMD
3
• Cutting-edge
cyberinfrastructure
• 360˚ video / audio
production
• Digital production and
programming
• Applied VR/AR research
• Second language and
cultural acquisition
• Science-of-learning
principles
• Assessment
• Technology in education
![Page 4: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/4.jpg)
Language learning and immersion
• For advanced skills, learners need to practice in
real-world contexts
• Immersive study abroad programs are beneficial (Davidson, 2010; Dewey, 2008; Linck et al., 2009; Segalowitz & Freed,
2004; Tare et al., in press)
4
• Challenge: Exposure to certain
mission scenarios may be too
costly, complex, rare, or dangerous
to allow real-life practice before
deployment
• Solution: Virtual training
![Page 5: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/5.jpg)
Technology affords multiple options
• Cinematic 360˚ film
• Digital virtual worlds
• Trade-offs • Static vs. adjustable content
• Ability to interact with characters and environment
• Degree of realism (e.g., language, movement)
![Page 6: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/6.jpg)
Why 360˚ cinematic film?
• Captures details:
o nuanced language
o nonverbal cues (e.g., micro-expressions, eye
gaze, body language)
• 360˚ spatial audio
• Affords viewer a sense of presence
(“being there”)
6
![Page 7: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/7.jpg)
VR design goals for prototype
7
• Scenario inspired by
intelligence agency
requirement
• Immersion in language
• Target listening ILR 2 – ILR 3
• Re-usable
• Supplement to instruction
![Page 8: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/8.jpg)
Embassy cocktail party prototype
• Needs-based content development:
o Interviewed Subject Matter Expert
oContracted native-speaking Russian actors
o Targeted, high-level content, loosely scripted
o Encouraged improvisation that fit with characters
• Designed for pedagogical exploitation
8
![Page 9: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/9.jpg)
2D video
9
360˚ video camera
Background
conversations
![Page 10: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/10.jpg)
360˚ video (flattened still)
10
![Page 11: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/11.jpg)
Instructor focus group
Five instructors of college-level Russian
“impressive”
“completely
blown away”
“felt like a fly on
the wall”
“would watch a
million times, it
is so rich”
“overwhelmingly
real”
![Page 12: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/12.jpg)
The promise of VR technology
12
![Page 13: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/13.jpg)
Does VR technology enhance learning?
• Digital VR = mixed results
Physical movements (Bailenson et al., 2008)
Medical procedures (Sutherland et al., 2006):
VR > no training, but = standard training
• 360˚ video = TBD (nothing published yet)
• Current study:
o Compare 360˚ video with traditional 2D video
o Outcome = L2 listening comprehension
o Examine potential mediators
13
![Page 14: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/14.jpg)
Experimental tasks
• 60-minute session:
o Baseline proficiency test
o Vocabulary test (pre and post)
o 8 minute video, viewed in three parts
• 2D condition
• 360˚ condition
o Listening comprehension (after each part)
o Experience questionnaires
o Language history questionnaire
14
![Page 15: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/15.jpg)
Measures
• Baseline proficiency (cloze test) • 25 blanks in 222-word text
• Listening comprehension o 12 multiple-choice questions
• Factual
• Inferential
• Opinion
• Tone
• Experience questionnaires • Presence
• Visual memory
• Open-ended
• Language history questionnaire 15
![Page 16: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/16.jpg)
Measures
• Vocabulary Knowledge Scale
o 20 target words
o Low frequency
(Wesche & Paribakht, 1996)
16
Self-report categories
I I don’t remember having seen this word before.
II I have seen this word before, but I don’t know what it means.
III I have seen this word before, and I think is means ________________.
IV I know this word. It means ________________.
![Page 17: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/17.jpg)
Participants
• 53 (28 female) native English speakers
with advanced L2 proficiency in Russian
o 360˚ VR condition: N = 27
o 2D condition: N = 26
17
Condition
VR 2D
L2 proficiency:
Cloze (out of 25) 13.2 (5.6) 13.4 (5.0) ns t = -0.16
Vocabulary pre-test (out of 80) 57.2 (9.5) 56.4 (7.7) ns t = -0.32
Age (years) 40.3 (9.5) 29.7 (8.8) p < .001 t = 4.14
* Similar L2 proficiency
* Younger participants in 2D condition
![Page 18: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/18.jpg)
Hypotheses
18
VR Presence Hypothesis 2
VR Listening
Comprehension
VR Vocabulary
learning
Hypothesis
1
VR Listening
Comprehension
Presence
Hypothesis 3
![Page 19: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/19.jpg)
Analysis
• Series of regression models
o “VR” dummy-coded factor: VR (1) vs. 2D (0)
o Bootstrap analysis to test for mediation effect
**Similar results when controlling for L2
proficiency; not reported here
19
![Page 20: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/20.jpg)
Distributions of outcomes
20
Listening
Comprehension
Vocabulary
Learning Presence
2D VR 2D VR 2D VR
![Page 21: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/21.jpg)
Hypothesis 1 results:
Training condition on Outcomes
21
VR Listening
Comprehension
VR Vocabulary
learning
b = 3.18 (0.54), p < .001
b = 1.21 (0.79), p = .13
R2 = .40
R2 = .04
Virtual Reality training condition improved listening comprehension but not incidental vocabulary acquisition
![Page 22: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/22.jpg)
Hypothesis 2 results:
Training condition on Presence
22
VR Presence
b = 1.26 (0.21), p < .001 R2 = .40
Virtual Reality condition associated with increased
sense of Presence
![Page 23: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/23.jpg)
Hypothesis 3 results:
Training condition and Presence
23
b = 3.03
95%CI (1.92, 4.20), p < .001
Bootstrap test of mediation effect indicates:
• Direct effect of VR on listening comprehension
• No indirect (mediated) effect via presence
VR Listening
Comprehension
Presence
b = 0.15
95%CI (-0.56, 0.84), p = .78
Presence
![Page 24: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/24.jpg)
Discussion
• Better listening comprehension found in
Virtual Reality (VR) training condition
• VR produced greater sense of presence
among participants “I felt like I was there - in same room”
“I found this to be surprisingly immersive, and it
mirrored a real-life situation very well.”
“It was like being in the room and being an active
participant in the environment”
24
![Page 25: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/25.jpg)
Discussion
• Even though presence did not directly
benefit L2 listening comprehension, the
increased sense of presence in VR is itself
a positive outcome:
o Increased realism
oGreater engagement
oHigher satisfaction with training
25
![Page 26: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/26.jpg)
Study conclusions
• Virtual training can enhance L2 listening
comprehension
• Cognitive mechanisms driving the benefits
of VR remain to be determined
26
![Page 27: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/27.jpg)
Other uses
• Integration of
speaking prompts
• Real-time assessment
for listening
• Interpretation training
27
![Page 28: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/28.jpg)
Acknowledgments
• Marsha Kaplan
• Laura Murray
• Richard Dabrowski
28
![Page 29: Practicing Russian Listening Comprehension Skills in ...€¦ · programming • Applied VR/AR research • Second language and cultural acquisition • Science-of-learning ... 4](https://reader034.vdocuments.site/reader034/viewer/2022043011/5fa6552a2c469433695a15d1/html5/thumbnails/29.jpg)
Thank you!
Ewa Golonka
Associate Research Scientist
Jared Linck
Research Scientist
[email protected] | @JaredLinck
www.casl.umd.edu @umdcasl
29