exploring the potential of speech recognition to support problem solving and reflection @ ec-tel...
DESCRIPTION
Talk at EC-TEL 2014 presenting joint work from Manolis Mavrikis, Beate Grawemeyer, Alice Hansen, and Sergio Gutierrez-Santos. London Knowledge Lab.TRANSCRIPT
![Page 1: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/1.jpg)
Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection
Manolis Mavrikis, Beate Grawemeyer, Alice Hansen, and Sergio Gutierrez-Santos
London Knowledge Lab
Wizards go to school in the elementary maths classroom
![Page 2: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/2.jpg)
Aim
• Enabling natural interaction with a learning platform through multimodal interaction - speech.
• Wizard-of-Oz study to – Collect realistic data – Inform the design of the system – explore children’s inner speech– reflection as a learning mechanism
![Page 3: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/3.jpg)
Voice interaction for learning• Educational research has shown benefits of verbalization of learning (1,2,3).
• More natural and efficient form of communication will have positive learning gains (4).
• Verbal communication in learning mathematics is important, as it is similar to learning a foreign language.
• Self-explanation can be beneficial for learning (5) as it can be seen as a tool to address student’s own misunderstandings.
• Speech can also provide cues for drawing inferences on students emotions and attitude toward the learning situation
![Page 4: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/4.jpg)
Wizard-of-oz study
![Page 5: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/5.jpg)
wizard
student
(1)
(6)(2)
(4)(3)
(5)
(5)
WOZ Setup
![Page 6: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/6.jpg)
Ecological validity
![Page 7: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/7.jpg)
Ecological validity
![Page 8: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/8.jpg)
Wizard-of-oz study
![Page 9: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/9.jpg)
Wizard-of-oz study
• Overall 60 students aged between 9 – 10 took part in a series of sessions with the iTalk2learn platform.
• 12/10 Students were wizarded for around 30 minutes.
• The students that were wizarded answered a questionnaire and took part in a focus group.
![Page 10: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/10.jpg)
Fractions Lab – exploratory learning environment
![Page 11: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/11.jpg)
Fractions Lab – exploratory learning environment
![Page 12: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/12.jpg)
Wizard’s script for support
![Page 13: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/13.jpg)
• Talk aloud• “Remember to talk aloud, and tell us what are you thinking”• “What is the task asking you to do?”• “Please think aloud, what are your thoughts or feelings?”
• Talk maths• “Can you explain that again using the terms denominator, numerator?”
• Affect • “It may be hard, but keep trying”• “If you find this easy, check your work and change the task”
• Problem solving • “You can’t add fractions with different denominators”
• Reflection• “What did you learn from this task?”• “What do you notice about the two fractions?”
Feedback types
![Page 14: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/14.jpg)
Wizard of Oz results
• 170 messages were sent to 10 students
Affect Talk aloud Talk maths Problem solving
Reflection0
10
20
30
40
50
60
70
80
![Page 15: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/15.jpg)
Wizard of Oz results
• Was the feedback immediately related to what the student said?
Affect Talk maths Problem solving Reflection0
10
20
30
40
50
60
70
80
NoYes
![Page 16: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/16.jpg)
Wizard of Oz results
• Student reacted?
Affect Talk aloud Talk maths Problem solving
Reflection0
10
20
30
40
50
60
70
80
NoYes
![Page 17: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/17.jpg)
Wizard of Oz results
• No significant correlation between the provision of feedback immediately after what the student said, and student’s reaction (r=.18, p>.05).
• However, there was a significant correlation if problem solving support was provided (r=.16, p<.05).
![Page 18: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/18.jpg)
Questionnaire5
4
3
2
1
helpful repetitive understand like think aloud
![Page 19: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/19.jpg)
Discussion• “Now you are talking”
– An ecological valid setting is important.
• “Sarcasm doesn’t become you”– Voice production is important.
• “Can the headsets look in my head?”– Students attribute higher abilities to the system.
• “Hmm – let me rephrase that “– Prompting students to use mathematic vocabulary might help to think
more carefully what they are saying.
• “Talking the talk”– Individual differences.
![Page 20: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/20.jpg)
Conclusion
• The results indicate that there is potential in expecting young students to think-aloud while interacting with educational technology.
• Even rudimentary ASD has the potential to support basic reflection on the learning task and of domain terminology.
• Future work includes identifying how interruptive the messages are and carefully selecting timings on reflective prompts.
![Page 21: Exploring the Potential of Speech Recognition to Support Problem Solving and Reflection @ EC-TEL 2014](https://reader035.vdocuments.site/reader035/viewer/2022062418/556c5e34d8b42acc228b51ef/html5/thumbnails/21.jpg)
References1. Askeland, M.: Sound-based strategy training in multiplication. European Journal of Special Needs Education 27(2), 201-217 (2012)
2. Borasi, R., Siegel, M., Fonzi, J., Smith, C.: Using transactional reading strategies to support sense-making and discussion in mathematics classrooms: An exploratory study. Journal of Research in Mathematics Education 29, 275-305 (1998)
3. Mercer, N., Sams, C.: Teaching children how to use language to solve maths problems. Language and Education 20(6), 507-528 (2007)
4. Mayer, R.E., Moreno, R.: Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist 38(1), 43-52 (2003)
5. Chi, M.: Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In: Glaser, R., (ed.) Advances in Instructional Psychology, 161-238 (2000)