TEST-DRIVEN GAMEDEVTESTS AUTOMATION AND GAME DEVELOPMENT

Team Leader / Senior Software EngineerWWW.GANYMEDE.EU

Konrad Gadzina

AGENDA

1. Why to automate tests?2. How to do it?3. Automated testing in

games4. Small demo5. TDD and legacy code6. Real-life working

experience7. How to learn?

THAT’S NOT THE SLIDE YOU’RE LOOKING FOR

a.k.a. why am I here?

0

WAIT, ISN’T IT ENOUGH TO PLAY THE GAME?

1

WAIT, ISN’T IT ENOUGH TO PLAY THE GAME?

1

MANUAL TESTING

What does it give us?

• checking playability and game feel

• finding exploits

MANUAL TESTING

What does it not give us?

• certainty while developing big changes

• quick feedback• full repeatability

MANUAL TESTING

Like board games vs computer games• in computer games you don’t have

to remember the rules• you can assume that they will be

predictable and repeatable• but, still, they cannot replace board

games – they are different

OK, AUTOMATE TESTS, BUT HOW?

2

WAIT, ISN’T IT A TESTER’S JOB?

Most people think that testing is only testers’ playground.Many programmers like to see it this wayThey don’t imagine how much they can do by themselves.

TYPES OF TESTS

There are many conventions, one of many popular divisions:• unit• functional• integration

WHAT TO USE?

“Unit tests tell a developer that the code is doing things right; functional tests tell a developer that the code is doing the right things.”

http://www.softwaretestingtricks.com/2007/01/unit-testing-versus-functional-tests.html

UNIT TESTS

• check correctness of specific elements in isolation

• enforce modular architecture• are written from the

programmer’s perspective• don’t check interactions

FUNCTIONAL TESTS

• check interactions between differents parts of the application

• written from the user’s perspective

INTEGRATION TESTS

• check correctness of communication with external systems

• are functional tests

TEST-DRIVEN DEVELOPMENT

Idea

• writing unit tests before writing production code

• writing as much as needed at the moment, nothing more

• very short iterations

TEST-DRIVEN DEVELOPMENT

Red-green-refactor mantra

1. Write test that fails2. Write production code to make

test pass3. Refactor

BEHAVIOUR-DRIVEN DEVELOPMENT

Behaviours

Creating behaviour scenarios that became integration tests.

Some say that „BDD is TDD done right”.

BEHAVIOUR-DRIVEN DEVELOPMENT

Communication

Core principle of BDD is creating a good communication between the dev team and the customer.

DSL is created – Domain Specific Language, that’s used in scenarios and is understable by both sides.

SO, WHAT’S WITH THOSE GAMES, AGAIN?

3

WHY WE DON’T TEST?

Seeming unprofitability of tests

“However, for most game code that is written once and then discarded, unit tests generally do not make sense. The cost to hire an room full of QA testers to verify everything at the end of the project is often less than the cost to create the automated tests.”

http://www.gamedev.net/topic/648772-unit-testing-ftw/

WHY WE DON’T TEST?

Potential problems with testing

“It is hard to write unit tests for code that is non-deterministic. If you have code involving random numbers, you won't be able to write a unit test that asserts an expected result.”

http://programmers.stackexchange.com/questions/250449/test-driven-development-for-complex-games

WHY WE DON’T TEST?

Why do you have problems?

Long-range thinking and modular architecture can solve many problems.

„BIG PLAYERS” AND TESTS

What great people say

"Whenever I come across a finicky looking bit of code now, I split it out into a separate pure function and write tests for it. Frighteningly, I often find something wrong in these cases, which means I'm probably not casting a wide enough net."

John Carmackhttp://gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php

„BIG PLAYERS” AND TESTS

What great people say

"I have never done real TDD. I make the excuse that real time stuff is harder to TDD for, but I know I don't do enough test."

John Carmackhttps://twitter.com/id_aa_carmack/status/459112425833246720

„BIG PLAYERS” AND TESTS

Job offers in games

UI / Scaleform Developer @ Rockstar:

“DESIRED (..)

● Experience with UNIT testing.”

Senior Software Engineer, C++ - Infrastructure @ Blizzard:

“Pluses (..)

● Experience designing and implementing unit tests“

UNITY TEST TOOLS

Free tools developed by Unity itself.It allows you to do:• unit tests• integration tests

http://u3d.as/65h

TALK IS CHEAP, SHOW ME THE CODE!

4

YASIC

Yet Another Space Invaders Clone

• recruitment game for Ganymede• JavaScript – Phaser + Jasmine• Behaviour-Driven Development

http://fenixb3.github.io/YASIC/

TDG PROJECT

Test-Driven Gamedev Project

• project started after workshops on Codepot conference

• Unity3D + Unity Test Tools• Test-Driven Development

https://github.com/FeniXb3/Test-Driven-Gamedev

WELL, HOW DOES TDD WORK WITH LEGACY CODE?

5

PRETTY AWESOME, AND YOU?

Changes in old code that we don’t even know are almost always risky.Regression is highly possible to occur, productivity of developers is lower.

EDIT AND PRAY VS COVER AND MODIFY

“To me, legacy code is simply code without tests.”

Michael C. FeathersWorking Effectively with Legacy Code

CHANGES IN LEGACY CODE

The safety net

1. Identify parts that have to be changed

2. Find place to put your tests3. Break dependencies4. Write tests5. Perform the change and refactor

YOU SAY GANYMEDE DOES IT?

6

YES, WE DOBut there are still some checkpoints before the final boss

TESTING IN GANYMEDE

What technology brings us

ActionScript:• FlexUnit + Mockolate• Made more friendly with

RobotlegsC++:• Catch or BanditUnity:• Unity Test Tools

TESTING IN GANYMEDE

What problems do we have?

ActionScript:• FlexUnit can’t run without

graphics• …so you can’t use it on CI server

in headless mode• and, well, it is Flash…

TESTING IN GANYMEDE

What problems do we have?

C++:• code we have in C++ is not new• …and it does not have any tests• …and many parts of it have

never heard of loose coupling

TESTING IN GANYMEDE

What problems do we have?

Unity:• you can’t unit test

MonoBehaviours directly• there are sealed classes hard to

mock• build on Jenkins running on Mac

fails when test uses NSubstitute• AssertComponent doesn’t work

on prefabs

TESTING IN GANYMEDE - UNITY

How to do it

To make yout code testable you can use:• MVC pattern• Humble Object Pattern

TESTING IN GANYMEDE - UNITY

How to do it – Humble Object Pattern

• create class that handles logic

• create tests for it• use its methods one-to-one

in MonoBehaviour• http://blogs.unity3d.com/2014/06/03/unit-te

sting-part-2-unit-testing-monobehaviours/

TESTING IN GANYMEDE - UNITY

How to do it – MVC

We use StrangeIoC – dependency injection and MVCS framework.It forces modular architecture.Ergo, it makes your code easier to test.http://strangeioc.github.io/strangeioc/

TESTING IN GANYMEDE - UNITY

How we do it – the whole picture

What we exactly do:• we create commands that are

triggered by signals• we test expected behaviour of

commands• we test services that are used

by commands

TESTING IN GANYMEDE - UNITY

How we do it – the whole picture

What we exactly do - continued:• we mock boundaries where

possible• we create integration tests

when needed• we have testers who can catch

bugs like multitouching where it should not be avaible

HOW TO DIG INTO TDD?

7

CODING DOJO

Meeting where group of people solve small programming problems (kata) using TDD.

There are two types of kata:• Prepared Kata• Randori Kata

CODING DOJO

Prepared Kata

• one person solve the whole problem

• the rest can suggest next tests• more friendly if that’s a first

contact with TDD

CODING DOJO

Randori Kata

• pair programming• change in pair after predefined

time• „baby steps”

OK, SUM IT UP!

∞

SUMMARY

• writing tests can take more time on development but less time on fixing

• TDD enforce you to not write useless code

• tests are your regression safety net• there are game companies who do it• we do it at Ganymede – it works!

www.ganymede.euwww.gamedesire.comwww.ganymedeacademy.com

kgadzina@ganymede.eu

THANKS FOR YOUR ATTENTION

@fenixb3