mobile video games testing principles - benjamin poirrier

1

MOBILE VIDEO GAMES

TESTING PRINCIPLES

Benjamin Poirrier

2

BEFORE WE START…

CONTENTS

• Software versus Game testing

• Mobile video game testing

• Mobile advertising testing

• Sum-up

3

CONTENTS




• Sum-up

Reference :

Johan Hoberg (Senior QA Engineer)

4

5

SOFTWARE AND GAME ARE SIMILAR

SIMILAR STUFF [1/3]

6

SOFTWARE TESTER VIDEO GAME TESTER

CHECKS THE APPLICATION’S

FUNCTIONALITIES FUNCTIONALITIES

SIMILAR STUFF [2/3]

7


CHECKS THE APPLICATION BASED ON

USER EXPERIENCE USER EXPERIENCE

SIMILAR STUFF [3/3]

8


PERFORMS CONTINUOUS TESTING

YES YES

9

YET DIFFERENT

DIFFERENCES [1/3]

10


NEEDS FOR GENERAL KNOWLEDGE AND

TECHNICAL SKILLS GAMING EXPERIENCE

DIFFERENCES [2/3]

11


HAS TO

WRITE TEST CASES FOLLOW GUIDELINES

DIFFERENCES [3/3]

12


WORKS IN A TEAM OF

1 OR 2 STAFF 10+ STAFF

CONTENTS




• Sum-up

References :http://www.gameloft-sea.com

https://www.linkedin.com/company/gameloft

https://www.facebook.com/Inside-Gameloft-South-East-Asia-529323844046869/?fref=nf

13

http://www.gameloft-sea.com/

https://www.linkedin.com/company/gameloft

https://www.facebook.com/Inside-Gameloft-South-East-Asia-529323844046869/?fref=nf

14

THE MAIN ACTORS

TEAMS

15

DEVELOPERS QUALITY ASSURANCE

• Producer

• Programmer

• Designer

• Artist

• Project manager

• Lead

• Senior Tester

• Junior Tester

RESPONSIBILITIES

16

DEVELOPERS QUALITY ASSURANCE

• Code the application

• Send build to QA

• Debug

• Test the application

• Report bugs to DEV

• Verify bugs

17

THE QA LIFE

QA MAIN TEST CASES

18

• Gameplay

• Sound

• Advertising

• Customer Relationship Management

• Event Tracking

QA MAIN CHALLENGES [1/4]

19

• Test scope / Checklist

• Porting / Device specifics

• Platform / Shops specifics


20




Project specifics

More and more features

Numerous updates

Dictionary checklist rules

Weekly updates


21




Master source to follow

Hundreds of devices

Ratio

GPU

Memory

Chipset


22




iOS

Android

Windows

Java

Shops

23

HOW TO OPTIMIZE ?

QA OPTIMIZATIONS [1/6]

24

• Perform quick / full rounds

• Follow devices’ hierarchy

• Cut low-risk test cases

• Focus on implemented features

• Randomize test cases


25






Project follow-up

Bug status

Resource analysis

Fix verification

Quick round of tests

Full round of tests


26






Master device

Submaster devices

Compatible devices

Auto-compatible

Publishing list update


27






Risk analysis

Previous test rounds

Experience

Bug fixing

Common sense


28






Master follow-up

Features anticipation

Bugs anticipation

% of implementation

Project planning


29






Intelligent walkthrough

Compatible

randomization

Last build full tested

30

WHAT ABOUT AUTOMATION ?

QA AUTOMATION [1/4]

31

• Use ACWT

• Duplicate tester behavior

• Analyze checklist rules

QA AUTOMATION [2/4]

32

• Use ACWT



Android Checker Web Tool

Web browser automation tool

Upload .apk

Check all package files

Quick and no human mistake

QA AUTOMATION [3/4]

33

• Use ACWT



Clone behavior on X devices

Employee performs test case

on 1 device only

Technical constraints = FAIL

QA AUTOMATION [4/4]

34

• Use ACWT



Thousands of checklist rules

Few can be automated

Global QA automation tool

currently work in progress

CONTENTS




• Sum-up

Reference :http://www.gameloft.com/advertising-solutions/

35

http://www.gameloft.com/advertising-solutions/

36

MOBILE AD PRODUCTS

IN-GAME AD FLOW

37

GAME

TRIGGER AD

REACH POINTCUT

CONTINUE GAME

IN-GAME AD FLOW

38

GAME

TRIGGER AD

REACH POINTCUT

CONTINUE GAME

Video

Banner / Interstitial

Rich Media

Incentivized or not

39

MOBILE AD TESTING


40

• Amount of devices / firmware

• Amount of pointcuts

• Duplicate bugs


41



• Duplicate bugs

Support all host game

device / firmware

Cannot test all


42



• Duplicate bugs

Many pointcuts to test

Trigger and rewards

Host related issues


43



• Duplicate bugs

Common bugs

Undoable

Waste of time

44

HOW TO OPTIMIZE ?


45

• Practice performance tests

• Use event tracking data

• Analyze interactions

• Reskin ads


46




• Reskin ads

Memory consumption

Loading times

High / low-end device

Master first


47




• Reskin ads

Analyze user behavior

Identify daily active users

(DAU)

Define successful KPI


48




• Reskin ads

Project experience

Ad bugs are device and

firmware related

Ad bugs are not host and

pointcuts related


49




• Reskin ads

Ads catalogue

Can re-use previous

template with new assets

Save lots of testing time

CONTENTS




• Sum-up

50

51

4 POINTS TO REMEMBER

[1] PROJECT FOLLOW-UP

52

• Cheap and efficient way to optimize testing

• Updated data allow Ad Hoc test

[2] KPI USE

53

• Spend efforts / time where / when necessary

• Event tracking is the key

[3] AUTOMATION PROMISES

54

• Still a long way to develop testing automation

• Video game hard to automate versus software

[4] COMMON SENSE

55

• Common sense cannot be reproduced by robots

• Still needed to apply efficient test cases