all rise! multiple perspectives on serious game evaluation - isaga 2012
DESCRIPTION
All rise! Multiple Perspectives on Serious Game Evaluation: Heide Lukosch, Delft University of Technology @ ISAGA 2012, RomaniaTRANSCRIPT
10-07-12
Challenge the future Delft University of Technology
All rise! Multiple perspectives on serious game evaluation
Heide Lukosch, Harald Warmelink, Rens Kortmann International Simulation and Gaming Association Conference 2012
10-07-12
Challenge the future Delft University of Technology
3 | 17
Why should we evaluate such a game?
How can we evaluate
such a game?
4 | 17
Structure
Starting points
Understanding of Serious Games
The Evaluation of Serious Games
Our Court Game Workshop
Multiple Perspectives on Serious Game Evaluation
The Contribution of Social Systems Theory
Conclusions and Discussion
5 | 17
Starting Points
Disagreements on a proper
evaluation method within the community
Evaluation is needed to increase
the quality of serious games
Which perspectives on serious game evaluation do exist?
Imag
e: for
rest
erbl
ogs.
serio
usga
min
g
6 | 17
Understanding of (Serious) Games
Games as tools to support
processes of
Organizational change
Decision Making
Learning
Policy making
What are preconditions and effects?
Images: CharliePapa simulation Game; Craft Game, Kenteq, NL
7 | 17
The Evaluation of (Serious) Games
Influenced by social sciences
Theory based evaluation
Design perspective
Analytical perspective
Focus on learning effects of a game
Images: CharliePapa simulation Game; TU Delft game session
8 | 17
The evaluation of (Serious) Games
Increasing interest in Serious Games Development
A lot of money is spent Still less is known about proper evaluation methods Often pre- and post- assesments are conducted to find proof for effectiveness Input – Proces – Output model
Images: CharliePapa simulation Game; TU Delft game session
9 | 17
Our Court Game Workshop
Role-play to facilitate a discussion Two opposite perspectives (plaintiff and defendant) representing our hypothesis: existence of economic and scientific perspective Group discussions
Image: DIGRA Court role play workshop
10 | 17
Our Court Game Workshop
Structured approach, applied to a role-play
Based on Kolfschoten/De Vreede, 2009
Iden
tific
atio
n
Introduction
Anal
ysis
Presentation of the positions
Summary
Alte
rnat
ives
Facilitated group discussion
Presentation of results Conclusions
De-Briefing on role on role
off role
off role
on role
11 | 17
Our Court Game Workshop
Dutch Civil Law
Game industry association
(plaintiff)
Game company
(defendant)
Expert Witnesses
Additional Material
Image: DIGRA Court role play workshop
12 | 17
Our Court Game Workshop
Items to discuss:
Types of measured variables Number of measuring instruments Number of respondents Use of theory Objectivity of evaluation
Image: DIGRA Court role play workshop
13 | 17
Multiple perspectives on Evaluation
Evaluation of effects problematic Evaluation of the design possible Value of Evaluation itself?
A third, „arts“ perspective emerged
Image: Dijk Patrouille game
14 | 17
The Contribution of Social Systems Theory
Arts
Sciences
Economics
Serious Games
observe
15 | 17
Conclusions
The way we look at evaluation of games and simulations in our research group Tools for change How can games influence change processes?
Image: Craft Game, Kenteq, NL
16 | 17
Conclusions
Multiple perspectives are related to distinct parts of modern society:
Science Economics Arts
Understood as paradigms, could they influence research on serious games?
Image: www.designingasociety.net
17 | 17
Discussion
Types of measured variables Number of measuring instruments Number of respondents Use of theory Objectivity of evaluation more items ?
Any more perspectives to consider?