an empirical analysis of ‘challenge’ as a motivational factor for educational games

An empirical analysis An empirical analysis of ‘challenge’ as a of ‘challenge’ as a motivational factor motivational factor

for educational for educational gamesgames

Conor LinehanConor Linehan**, Ben Kirman, Ben Kirman**, , Bryan RocheBryan Roche##

**Lincoln Social Computing Research Lincoln Social Computing Research CentreCentre

##NUI MaynoothNUI Maynooth

IntroductionIntroduction

BackgroundBackground– Games as tools for educationGames as tools for education– ‘‘Appropriate challenge’ and flowAppropriate challenge’ and flow

Measuring motivation (Behaviour Measuring motivation (Behaviour Analysis)Analysis)

MethodologyMethodology ResultsResults DiscussionDiscussion

Games as tools for Games as tools for educationeducation

Games are good at 1) engaging players and 2) Games are good at 1) engaging players and 2) motivating them continue playing.motivating them continue playing.

Delivering educational programmes through Delivering educational programmes through game-like structures should allow for engaging game-like structures should allow for engaging and intrinsically motivating education.and intrinsically motivating education.

But we don’t know what game-like features are But we don’t know what game-like features are necessary……necessary……

Moreover, there’s very little evidence to show Moreover, there’s very little evidence to show that games that games AREARE effective as educational tools. effective as educational tools.

should

Games as tools for Games as tools for educationeducation

O’Neill, Wainess & Baker (2005) - review - O’Neill, Wainess & Baker (2005) - review - despite the thousands of articles available, despite the thousands of articles available, only 19only 19 presented a rigorous quantitative presented a rigorous quantitative or qualitative analysis of educational or qualitative analysis of educational outcomes from computer games. outcomes from computer games.

……..And the results of these contradictory!..And the results of these contradictory! A lot of myths, rules-of-thumb, A lot of myths, rules-of-thumb,

presumptions. Not very much evidence.presumptions. Not very much evidence.

Appropriate Challenge & Appropriate Challenge & FlowFlow

One presumption is ubiquitous – that games One presumption is ubiquitous – that games must offer ‘appropriate challenges.’must offer ‘appropriate challenges.’

……suggests that players will not be suggests that players will not be motivated to play a game that they do not motivated to play a game that they do not find challenging.find challenging.

So, games can’t be too easy or too difficult So, games can’t be too easy or too difficult or players will lose interestor players will lose interest

Related - a Related - a flow stateflow state is said to occur when is said to occur when the player experiences appropriate the player experiences appropriate challenge in a game (i.e., it is neither too challenge in a game (i.e., it is neither too hard and frustrating or too easy and hard and frustrating or too easy and boring).boring).


But no studies have experimentally But no studies have experimentally examined what challenge & flow mean in examined what challenge & flow mean in games games

Scoring 100%? Scoring 100%? < 100%? < 100%? (How much less than 100%?)(How much less than 100%?) OROR ……..entirely subjective and dependent on ..entirely subjective and dependent on

each players’ unique historyeach players’ unique history??


The games requirements appear to The games requirements appear to contradict the educational requirementscontradict the educational requirements

……..the learner must have a goal of reaching ..the learner must have a goal of reaching 100% mastery in educational games 100% mastery in educational games

Not congruent with ‘Appropriate Challenge’Not congruent with ‘Appropriate Challenge’– If players prefer not to reach mastery at a game, If players prefer not to reach mastery at a game,

(but prefer some less stringent but appropriate (but prefer some less stringent but appropriate challenge), then forcing them to do so may lead to challenge), then forcing them to do so may lead to a loss of player motivation.a loss of player motivation.

– If players like to reach mastery – then this idea of If players like to reach mastery – then this idea of flow and appropriate challenge seem unnecessaryflow and appropriate challenge seem unnecessary

Measuring motivationMeasuring motivation

One crucial problem - there does not One crucial problem - there does not appear to be any existing objective appear to be any existing objective measure of motivation in games. measure of motivation in games.

Behaviour analysis - oBehaviour analysis - of particular f particular relevance, relevance, operant choice proceduresoperant choice procedures (i.e., Herrnstein, 1961) provide an (i.e., Herrnstein, 1961) provide an objective and quantitative means for objective and quantitative means for evaluating game players’ preference for evaluating game players’ preference for games or game elements. games or game elements.

Operant choice proceduresOperant choice procedures

Simple ideaSimple idea Individuals tend to divide their time and Individuals tend to divide their time and

effort between two or more simultaneously effort between two or more simultaneously available behaviour options proportional to available behaviour options proportional to the reinforcement that is contingent on the reinforcement that is contingent on each (Herrnstein, 1961). each (Herrnstein, 1961).

Games - players will choose to play games Games - players will choose to play games that offer the highest level of reinforcement that offer the highest level of reinforcement

This choice is our This choice is our measure of preferencemeasure of preference, , motivation to continue playingmotivation to continue playing

MethodologyMethodology

We allowed players to play We allowed players to play six simple six simple games games before asking them to choose before asking them to choose which one to re-playwhich one to re-play

Stage 1: Play all six games in randomised Stage 1: Play all six games in randomised order.order.

Stage 2: Choose & Play one of them againStage 2: Choose & Play one of them again Within each game:Within each game:

– Had to choose whether to ‘save’ or ‘destroy’ Had to choose whether to ‘save’ or ‘destroy’ each game character in each game.each game character in each game.

– Different stimuli presented across games.Different stimuli presented across games.– Score increased on correct response.Score increased on correct response.

Slow Medium speed

Fast

2 stimuli 2-slow 2-med 2-fast

6 stimuli 6-slow 6-med 6-fast

A spectrum of games, from those that are slow A spectrum of games, from those that are slow and easy, to those that are fast and complex.and easy, to those that are fast and complex.

Should allow for a distribution of player success.Should allow for a distribution of player success.

MethodMethod

Remember, for educational gamesRemember, for educational games If we find that players prefer not to reach If we find that players prefer not to reach

mastery at a game, forcing them to do so mastery at a game, forcing them to do so may lead to a loss of player motivation.may lead to a loss of player motivation.

If we find that they like to reach mastery – If we find that they like to reach mastery – then this idea of flow and appropriate then this idea of flow and appropriate challenge seem unnecessarychallenge seem unnecessary

Q1 – Do players enjoy reaching mastery at Q1 – Do players enjoy reaching mastery at a game?a game?

Q2 – If they don’t enjoy mastery, is there Q2 – If they don’t enjoy mastery, is there an ‘appropriate’ level?an ‘appropriate’ level?

MethodMethod

By looking at which game was chosen in By looking at which game was chosen in stage 2, and what the player previously stage 2, and what the player previously scored on that game in stage 1, we can scored on that game in stage 1, we can answer these questions.answer these questions.

Recruited fourty-three participants (24 Recruited fourty-three participants (24 male, 19 female), all aged 18-25.male, 19 female), all aged 18-25.

3 eliminated for not attending to 3 eliminated for not attending to experimental taskexperimental task

ResultsResults

Games chosen by type

0

2

4

6

8

10

2-slow 2-med 2-fast 6-slow 6-med 6-fast

Game type

Nu

mb

er

of

part

icip

an

ts

wh

o c

ho

se t

his

gam

e

Percentage correct on chosen game

0 10 20 30 40 50 60 70 80 90

Percentage of correct responses

1

2

3

4

5

6

7

8

9

10

11

Nu

mb

er o

f p

art

icip

an

ts

ResultsResults

11 chose to re-play games in which they 11 chose to re-play games in which they scored over 90% scored over 90%

9 chose to re-play games in which they 9 chose to re-play games in which they scored between 80% and 90% correct. scored between 80% and 90% correct.

Thus, half of all participants chose to re-Thus, half of all participants chose to re-play games in which they attained a score play games in which they attained a score of 80% or above.of 80% or above.

Percentage correct on chosen game

0 10 20 30 40 50 60 70 80 90

Percentage of correct responses

1

2

3

4

5

6

7

8

9

10

11

Nu

mb

er o

f p

art

icip

an

ts

ResultsResults

Very few (10%) participants chose to re-Very few (10%) participants chose to re-play games in which they attained scores play games in which they attained scores of 46% or less. of 46% or less.

The remaining 40% chose games in which The remaining 40% chose games in which they scored between 46% and 72%. they scored between 46% and 72%.

None chose to re-play a game in which None chose to re-play a game in which they scored between 72% and 82% in they scored between 72% and 82% in stage 1.stage 1.

ResultsResults

Q1 – Do players enjoy reaching mastery at Q1 – Do players enjoy reaching mastery at a gamea game– 11/40 chose games 90%+11/40 chose games 90%+– 20/40 chose games 80%+20/40 chose games 80%+

Q2 – If they don’t enjoy mastery, is there an Q2 – If they don’t enjoy mastery, is there an ‘appropriate’ level?‘appropriate’ level?– There does not appear to be one level of There does not appear to be one level of

success that was generally preferred, success that was generally preferred, with 90%+ surprisingly being the most with 90%+ surprisingly being the most preferred.preferred.

DiscussionDiscussion

If we found that players preferred not to If we found that players preferred not to reach mastery at a game, forcing them to reach mastery at a game, forcing them to do so may lead to a loss of player do so may lead to a loss of player motivation.motivation.

But this was not the case for 50% of But this was not the case for 50% of participantsparticipants

So forcing players of educational games to So forcing players of educational games to reach mastery seems okreach mastery seems ok

Ed. programmes that require mastery Ed. programmes that require mastery should not be ignored just because of should not be ignored just because of games lit on appropriate challengegames lit on appropriate challenge

DiscussionDiscussion

More generally:More generally: The concept of ‘appropriate challenge,’ as The concept of ‘appropriate challenge,’ as

crucial to the motivation inspired by a crucial to the motivation inspired by a game, must be reconsidered.game, must be reconsidered. – Half of participants liked games in which there Half of participants liked games in which there

did not appear to be a challengedid not appear to be a challenge– The rest of the scores were widely distributedThe rest of the scores were widely distributed

In designing educational games - aiming to In designing educational games - aiming to provide all participants with an appropriate provide all participants with an appropriate level of game challenge will apparently level of game challenge will apparently alienate rather a lot of potential players. alienate rather a lot of potential players.

DiscussionDiscussion Behaviour analysis explanationBehaviour analysis explanation:: Operant choice procedures tell us that Operant choice procedures tell us that

players will choose the most reinforcing players will choose the most reinforcing game.game.

Very few chose the game on which they Very few chose the game on which they scored the most points.scored the most points.

Although 50% did choose games on which Although 50% did choose games on which they previously recorded a high (80%+) they previously recorded a high (80%+) score.score.

So, not as simple as saying the points acted So, not as simple as saying the points acted as reinforcers.as reinforcers.

What What ARE ARE the reinforcers in games??the reinforcers in games??

ContactContact

Dr. Conor LinehanDr. Conor Linehan Lincoln Social Computing (LiSC) Research Lincoln Social Computing (LiSC) Research

Centre, School of Computer Science, Centre, School of Computer Science, University of Lincoln, Brayford Pool, University of Lincoln, Brayford Pool, Lincoln, LN6 7TSLincoln, LN6 7TS

[email protected]@lincoln.ac.uk +1522 837084+1522 837084

an empirical analysis of ‘challenge’ as a motivational factor for educational games

Entertainment & Humor