designing game-like activities to engage adult learners in higher education

Designing game-like activities to engage adult learners inhigher education

A. Mora, E. Planas & J. Arnedo-Moreno

Universitat Oberta de Catalunya

{amoraca|eplanash|jarnedo}@uoc.edu

November 2, 2016

A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 1 / 24

Index

1 Introduction

2 Description

3 Evaluation

4 Discussion

5 Conclusions


Introduction

Highly motivated students have better chances of acquiringcompetencies in higher education

Many studies are about how encourage the use of game designprinciples for student engagement

Most of them are focused on the 20-25 age bracket in face-to-facefull-time studies

UOC, a completely online-based university

Demography: >30 years old (64%), working full-time (95%)


Background

Gamification: ”the use of game design elements in non-gamecontexts” (Deterding et al., 2011)

Educational experiences in the literature present diverse results

It is a delicate design process: ad-hoc versus frameworks (Mora et al.,2015)

There is not enough evidence about engagement of adults learners inhigher education

Will adult learners be motivated in solving formative activitieswith game-like course design?


Description: foundations

Requirements Engineering (RE) scope

Computer Engineering grade (optional subject)

Study of the activities involved in the requirements engineeringprocess

Time-line comprised in fifteen school weeks

Evaluation based on three mandatory assignments anddiscussions


Description: foundations

Problem to solve

Lack of motivation in solving non-evaluative activities (0% last ed.)

Low sense of comradeship between online students

Expected behaviours

EB1: Motivated students to solve formative (not graded) activities

EB2: Increase of the sense of kinship between e-learners

Motivation types

MT2: Disciplined-materialist students (20.6%)

MT3: Persistent-idealist students (28.8%)


Description: design principles

A design framework that aims to structure the design process


Description: design principles

SPARC analysis

Sense: coherence with the learning process

Purpose: a clear intention form the instructors standpoint

Autonomy: let students to make choices

Relatedness: positive impact on the rest of students and the course

Competency: assurance that students are able to master the tasks


Description: the rules

Inspired in a reward-based crowd-funding model

Formative tasks: type (individual/teamwork), challenge and expirable

Phases: on boarding (simple tasks as tutoring), mid-game (regulartask development) and endgame (final tasks to goal achievements).

Task-activation timeline: 5-10 weekly

Expected average performance: 2-3 tasks per student.

All students could benefit even if they have not participated



Table: Example of formative tasks

Id Task description Points

IND

TEAM

CH

EXP

PhaseT1 Convince at least X students to partic-

ipate in the activity1 x x on boarding

T2 Resolve an exercise proposed in thelearning resources

1 x mid-game

T3 Resolve a doubt posted in the courseforum from another student at most 2days after posting

1 x x mid-game

T4 Commit to obtain the highest grade inan evaluative activity of the course

1 x x x mid-game

T5 Resolve an exam model 1 x x mid-gameT6 Correct the resolution of an exam de-

veloped by another student1 x x mid-game

T7 Answer a survey about the course 2 x end-game



Table: Example of rewards

Id Academic-reward Total grade tobe achieved

R1 Have extra days to deliver the evaluative activity beingcarried out at the time of reaching the score

10

R2 Evaluate the current activity on 12 points instead of 10 25R3 Have extra days to deliver the rest of the evaluative ac-

tivities of the course40

R4 Provide the students with examples of resolved exams 60R5 Do not take into account the worst exercise during the

evaluation of the exam80


Description: the metaphor

An Agile software development project (Scrum)


Description: the tool

Trello, a web-based project management application based on Agileparadigm

It provides a highly interactive experience (Drag & Drop)

Each task is represented by a card in the board,

Widely accepted in other gamified experiences (Gonzalez & Mora,2014)


Evaluation: methodology

Methodology for evaluation inspired in ”Four Level Model”(Kirkpatrick, 1967): reaction, how learners respond to theapplied process

Remember: the purpose is not the improvement of marks fromprevious courses

Instruments: task manager, forum and questionnaires

Managing quantitative & qualitative data


Evaluation: population (quantitative)

The study was conducted since September 2015 to February 2016

Two groups of the Requirements Engineering course

Demographic, N=94; Age range=22-53; M=34.07; Me=34 andSD=6.88; 91/94 male

Sample: 60 students enrolled voluntarily into the game-like activity(63.82% of total)

Representative sample


Evaluation: performance (quantitative)

First group 62/100 points, second group 80/100

Drop out rate of 0%

26.67% of passive participants (0 task)

18.34% of lower active participants (<2 tasks)

54.99% of higher active participants (>= 2 tasks)

An increment of 35% of posts from previous editions


Evaluation: growth (quantitative)


Evaluation: correlation analysis (quantitative)

Pearson’s coefficient (age, group, and accomplished tasks)

Positive medium-moderate correlation between ”age of activeparticipants” and ”accomplished tasks”


Evaluation: questionnaire (qualitative)

Table: Design questionnaire (n=32)

Question Scope AcceptanceDo you think the tasks have been proposedmake sense in the context of this subject?

Sense 93.75%

Do you think it has been achieved the goal ofencouraging the process of student learning?

Purpose 68.75%

Do you think it has been achieved the goalof encouraging student participation in activ-ities?

Purpose 75%

Have you freely chosen tasks you wanted todevelop?

Autonomy 93.75%

Do you think it has been achieved the goal ofpromoting teamwork?

Relatedness 56.25%

Do you think that the activity has createdfellowship feeling with other students ?

Relatedness 53.13%

Do you think it has been achieved the goal oflearning about project management tools?

Competency 90.63%


Discussion (individual overview)

The majority of the course students (63.82%) voluntarily participate

Half of the students revealed to be intrinsically motivated

More than a half of them participated in two or more tasks(expectation).

Motivating adult learners in solving non-graded formativeactivities was reached (EB1)


Discussion (group overview)

No behaviour difference between groups until two weeks beforeending (R4 & R5 achievements)

Appearance of improvised leaders (role of motivators and organizers)

25.70% of the total posts in forum were directly related to thegame-like experience

Increment of the sense of kinship between students, althoughto a lesser extent than desired (EB2)


Conclusions

It has been presented the design and analysis of a gameful e-learningexperience with adult on-line learners

SPARC framework was proposed as designed approach

Objective: Improving the student motivation in two different ways

A total of 60 students voluntarily joined the experience

Quantitative and qualitative analysis was conducted

”Relatedness” dimension need further improvement

Further work: personalization of gamification (MT1...MT4)


References

S. Deterding, R. Khaled, L. E. Nacke, and D. Dixon.

Gamification: Toward a definition.

In CHI 2011 Gamification Workshop Proceedings, pages 12–15, 2011.

A. Mora, D. Riera, C. Gonzalez, and J. Arnedo-Moreno.

A literature review of gamification design frameworks.

In proceedings of Games and Virtual Worlds for Serious Applications (VS-Games),2015 7th International Conference on, pages 1–8. IEEE, 2015.

C. Gonzalez and A. Mora.

Methodological proposal for gamification in the computer engineering teaching.

In Computers in Education (SIIE), 2014 International Symposium, pages 29–34.IEEE, 2014.

D. L. Kirkpatrick.

Evaluation of training.

1967.


Thank you!!!
