learning analytics in serious games
DESCRIPTION
Talk at VS-GAMES 2012 about learning analytics in educational games. Ángel Serrano-Laguna, Javier Torrente, Pablo Moreno-Ger and Baltasar Fernández-Manjón. Tracing a little for big Improvements: Application of Learning Analytics and Videogames for Student Assessment #galanoeTRANSCRIPT
![Page 1: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/1.jpg)
Tracing a little for big Improvements: Application of Learning Analytics and Videogames for Student Assessment
Baltasar Fernández-Manjó[email protected]
e-UCM research groupwww.e-ucm.es
VS-GAMES 2012, Genoa, Italy
![Page 2: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/2.jpg)
Educational videogames
• Teachers are starting to use educational videogames in order to explore new ways to educate their students – Still low adoption
• Videogames left as “low-weight” complementary content– Mainly used for motivational purposes– No actual impact on the final mark
![Page 3: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/3.jpg)
Serious games assessment
• No many serious games include in-game evaluation
• Serious games with integrated assessment usually rely in Q&A structures
• … but games produce a lot of data that can be analyzed with educational/assessment purposes
• The “box” game should be open …(white box)
![Page 4: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/4.jpg)
What do we analyze?
• Every game is very different– But we can group them by:• Game mechanics• Game genre• …..
– There some regularities that can be exploited
• Can we define a simple set of universal traces to analyze?
![Page 5: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/5.jpg)
Start, end, quit game traces
• Start game: whenever a student launches the game– Information: How many students played the game, who they
were and when they played.• End game: whenever a student successfully the game.
– Information: who accomplished the goals established for the game
– Does the optimal goal attain?• Quit game: whenever a student quits the game, before
finishing– Information: who abandoned the game before finishing it,
and with the appropriate context, where he quitted.
![Page 6: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/6.jpg)
Phase changes
• Usually, games are divided in phases.– In an educational videogame, these phases can
mark several educational sub-goals.• Tracing phases changes can be used to:
• Identifying most time-consuming phases• Understand how each part of the game is being
accessed (if the phase exploration sequence is not linear)
• … helping to improve the educational game
![Page 7: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/7.jpg)
Significant variables
• Games rely on variables to represent their state– Some of those variables can be relevant for the
assessment– Logging when and with which values these
variables are updated
![Page 8: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/8.jpg)
User interaction
• Raw user interaction (mouse clicks, screen touches, keys pressed…) can be used to retrieve some useful information– Heat maps:• To detect game design flaws
• If all user interaction is logged, the whole game play could be reproduced
![Page 9: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/9.jpg)
Some requirements
• Most of games are black boxes.– No access to what is going on during game play.
• We need access to game “guts”
• Or… the game must communicate with the outside world, using some logging framework– Not applicable to COTS games (yet)
![Page 10: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/10.jpg)
Gleaner: Game Learning Analytics for education research
• Framework oriented to capture game traces
![Page 11: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/11.jpg)
An example: Lost in Space <XML>
![Page 12: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/12.jpg)
An example: Lost in Space <XML>– Game for teaching XML– Played by students in the classroom• 1 to 2 hours playing• 2 hours defining new levels
– Uses Gleaner to log students interactions
![Page 13: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/13.jpg)
Metrics in “Lost in Space <XML>”
• Start and end game• Phases changes• Significant variables:– XML commands introduced by the students– Phases scores
• User interaction– Clicks on help button
![Page 14: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/14.jpg)
Some early results
• Real time metrics– Teacher could see student progress in real time from
its computer– At the end of the class, he knew how many students
had completed the game• Post-analysis– Most common pitfalls in XML commands where
detected– Interactions with the help button indicated those
phases where students had more trouble
![Page 15: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/15.jpg)
Yet Another eAdventure example
• “The big Party”– Game to teach students with disabilities about
habits on their daily life
![Page 16: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/16.jpg)
Metrics in “The big party”
• Implemented with a add-on to an eAdventure game
• Information collected– User interactions• All mouse interactions (including movement)
– Phases changes– Times spent in every phase– Order in phase discovery
![Page 17: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/17.jpg)
Some results
• Heatmaps showing elements with most interaction
• Tracing ALL interactions allow us to reproudce the entire game play
![Page 18: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/18.jpg)
Conclusions
• With simple traces, we can know a lot of about what is going on in our educational games
• We can provided real-time feedback to teachers and students, and significantly improve the educational process
• This process can also help to improve the game quality
• However, rigorous assessment should be based on a deeper data analysis. This is only a first step.
![Page 19: Learning Analytics in serious games](https://reader035.vdocuments.site/reader035/viewer/2022062418/55495174b4c905f24e8b4b02/html5/thumbnails/19.jpg)
• http://e-adventure.e-ucm.es– New 1.5 version (include Chinese, Rusian and Brasilian version)– Multiplatform (Windows, Linux, Mac)– Videos (also in youtube in the eAdventureUCM channel)– Tutorials – Games (that you can reuse and modify)
• Open source code– Sourceforge.net– You can contribute (e.g. coding, eA translation)
• Baltasar [email protected]