x-construction lite: hci evaluation
DESCRIPTION
Presentation of a usability study a group of MSc students at Sussex University (including myself) carried out on the Android game, X-Construction LiteTRANSCRIPT
Tom Hume Human-Computer Interaction, assignment 1
Text
TitleX-Construction Lite: HCI evaluationAndy Keavey, Mariana de Rojas-Morao,
Merve Yildirim, Queen Ajifa Ododo, Tom Hume
Tom Hume Human-Computer Interaction, assignment 1
Text
Title
Screenshot of page from Android Marketplace
Tom Hume Human-Computer Interaction, assignment 1
Overview of gameplay
First screen of instructions (animates) Second screen of instructions
Game-play screen Results of play screen
Tom Hume Human-Computer Interaction, assignment 1
1. Expert review1.1. Cognitive walkthrough, written up1.2. Evaluation vs Nielsen heuristics
2. User testing2.1. Pre-test questionnaire2.2. User test of first level (video recorded)2.3. Interview with user whilst reviewing recording2.4. Post-test questionnaire
Evaluation approach
Tom Hume Human-Computer Interaction, assignment 1
User evaluation
8 test subjects recruited
(all Android owners, familiar with UI)
Tom Hume Human-Computer Interaction, assignment 1
Results of cognitive walkthrough
Issue # reportsZoom functions misused ✓ ✓ ✓Errors placing girders ✓ ✓ ✓Misunderstanding help messages ✓ ✓ ✓Unsure how to proceed ✓ ✓Game crashed on start ✓Problem starting play ✓Confused by pop-up adverts ✓ “I successfully drew some
girders, but I struggled a bit because they were going the wrong direction most of the time”
“Some of my girders went out of view at the top of the screen. I used the zoom out gesture consistent with Google Maps and could see a much better view of the task”
Tom Hume Human-Computer Interaction, assignment 1
Results of heuristic evaluation
Agreement DisagreementMatch between system & real worldConsistency & standardsFlexibility & efficiencyAesthetics & minimalism
User control & freedomError preventionRecognition over recall
Tom Hume Human-Computer Interaction, assignment 1
Seconds to complete first level
Results of quantitative data
0
75
150
225
300
0 1 2 3 4
# attempts to complete first level
Tom Hume Human-Computer Interaction, assignment 1
Results of questionnaire data
0
10
20
30
40
8 7 5 4 6 1 3 2
Age of test subjects
25%
75%
Female Male
0
2
4
6
8
1 2 3 4 5 6 7 8
Estimated vs actual fun
Estimated Actual
38%
63%
Android gamer Not android gamer
Tom Hume Human-Computer Interaction, assignment 1
Results of user testing
Issue # usersMisunderstood girder placement ✓ ✓ ✓ ✓ ✓ ✓Didn’t understand help/instructions ✓ ✓ ✓ ✓ ✓Mixed up replaying/restarting level ✓ ✓ ✓ ✓Couldn’t get the train started ✓ ✓ ✓Panned unintentionally ✓ ✓Couldn’t tell they were complete ✓Couldn’t find girder removal tool ✓Confused by an advert ✓Misunderstood tool instructions ✓Couldn’t find instructions again ✓
Tom Hume Human-Computer Interaction, assignment 1
Key issue: misunderstood girder placement
Tom Hume Human-Computer Interaction, assignment 1
Key issue: mixed up replaying/restarting level
Tom Hume Human-Computer Interaction, assignment 1
Good news• Most players complete the first level, first time• Most players enjoy the game more than they thought• Many expert evaluators are now regular players
Bad news• 75% players, 3/5 experts had issues with placing girders• 62% players don’t read help, 3/5 experts misunderstand it• 50% players mix up replaying and restarting the level• 37% players, 2/5 experts have trouble starting their train • 20% players pan unintentionally, 3/5 experts had zoom issues
Conclusions
Tom Hume Human-Computer Interaction, assignment 1
1. Warn players on the first level when they place a girder which doesn’t finish attached to an end-point;
2. Clarify language in help messages and screens;
3. Improve the iconography, labelling or placement of the “replay failure” and “restart level” icons to reduce confusion between them;
4. Teach players to use the “play” button by animating it on the first level when they have a workable bridge;
5. Allow pinch-and-zoom to zoom into the bridge during play.
Recommendations
Tom Hume Human-Computer Interaction, assignment 1
Critique of methods
Hard to record video in sunlight
Users were self-selecting, to a degree
Slight inconsistencies in questionnaire usage
Tom Hume Human-Computer Interaction, assignment 1
Questions?