testing. rhs – soc 2 testing so far, we have not really created tests in a systematic manner...

Testing

RHS – SOC 2

Testing

• So far, we have not really created tests in a systematic manner

• Testing has been more like ”probing”

• Making a thorough test is often very labor-intensive – many SW companies employ more testers than developers…

• Once it is done, it is a very useful tool to use during development

RHS – SOC 3

Testing

• The ideal test:

• When the test is passed, the product is ready for delivery!

• Ideal – but (almost) impossible– Number of test cases is often very large– How does one test a GUI?– Is functional correctness all that matters?

RHS – SOC 4

Testing

• In general, test goes hand in hand with requirement specification

• If we cannot specifiy what the software should do, how can we test it…?– ”It must be easy to use the software”– ”The software should respond quickly”– ”The software must never crash”

RHS – SOC 5

Testing

• Requirements are speci-fied at many leves– Business level– User level– Functional level

• So we must also perform tests at many levels

RHS – SOC 6

Testing

http://upload.wikimedia.org/wikipedia/commons/9/96/V-model.JPG

RHS – SOC 7

Unit testing

• A Unit Test is aimed at testing a well-defined code module, in Java usually a single class

• Unit tests are at the functional level– Define test cases in terms of input to class

methods (public and private)– Define the expected output for each case– Run the test– Compare expected and actual output

RHS – SOC 8

Unit testing

• Unit testing is – more or less – what we have done so far, but informally

• NetBeans can create a more ”formal” unit test framework for a project

• Relies on a Java framework called Junit (see www.junit.org)

http://www.junit.org/

RHS – SOC 9

Unit testing in NetBeans

• Consider our ”classic” BankAccount class, with three methods:– deposit– withdraw– getbalance

• Having created the class, we can now create a unit test for the class

RHS – SOC 10


RHS – SOC 11


RHS – SOC 12


RHS – SOC 13


• There are quite a lot of options to choose from when generating a test class…

• For now, just leave them as-is

• When pressing Finish, a unit test class is generated for us, called BankAccount-Test (just choose ”Junit 4.x”)

• The test class is placed under Test Packages

RHS – SOC 14


• The generated test class does look a bit complex (remember we chose all options)

• However, we are free to edit it!

• Remove whatever you do not need

• NetBeans can only generat a ”skeleton” for the test class – we must complete it

RHS – SOC 15


@BeforeClass

public static void setUpClass() throws Exception

{}

@AfterClass

public static void tearDownClass() throws Exception

{}

RHS – SOC 16


• The two methods setUpClass and tearDownclass allows us to include any actions needed before and after running all the test class methods, respectively– Dependencies to other classes– Database connection– Etc.

• Is often not used – then delete it!

RHS – SOC 17


@Before

public void setUp() {}

@After

public void tearDown() {}

RHS – SOC 18


• The two methods setUp and tearDown allows us to include any actions needed before and after running each of the test class methods, respectively– Initialising/resetting variable values– Cleaning up data structures– Etc.

• Is often not used – then delete it!

RHS – SOC 19

Unit testing in NetBeans@Test

public void testGetBalance()

{

System.out.println("getBalance");

BankAccount instance = new BankAccount();

int expResult = 0;

int result = instance.getBalance();

assertEquals(expResult, result);

// TODO review the generated test code and

// remove the default call to fail.

fail("The test case is a prototype.");

}

RHS – SOC 20


• Notice that a test method does not return a value (true/false)

• Instead, so-called assertions are used during the test

• An assertion can succeed or fail

• A failed assertion throws an exception, and the test case is considered failed

RHS – SOC 21


• Examples of assertions:– assertEquals(expectedValue, ActualValue)

– assertTrue(condition)– assertFalse(condition)– assertNotNull(object)– assertNull(object)– assertSame(object, object)– assertNotSame(object, object)– fail() // ALWAYS fails

RHS – SOC 22


• If you inspect the generated test code, you will find that it is not very useful

• We must – almost always – implement the body of the test methods ourselves

• We are free to add more test methods than those initially generated – the test framework will run them automatically

RHS – SOC 23


• Once the test methods have been defined properly, we can run the test

• Choose Run | Test Project, or just press Alt + F6

• Result of test is displayed in the output window, with indicative colors

RHS – SOC 24


RHS – SOC 25

Exercises

• Create a new Java project called BankAccount, and implement it as described in the presentation

• Generate a Unit test class for BankAccount, as described in the presentation

• Now change the code in the generated test class, in order to make it a useful test for the BankAccount class. Note that this may involve not only changing the code in the test methods, but perhaps also adding code to some of the setUp/tearDown methods

• Run the test, and keep changing the source code and/or test code until all tests are passed (all lights are green ) – how confident do you now feel about the correctness of the BankAccount class?

• Review the test – did you have to make some assumptions during implementation of the test, that were perhaps questionable?

RHS – SOC 26

Unit testing details

• Using assertions for testing is the most common way of performing a unit test (also see http://junit.org/junit/javadoc/4.5)

• However, there are other aspects of test– Exceptions– Execution time

• These can also be tested with Unit test

http://junit.org/junit/javadoc/4.5

RHS – SOC 27


• In some cases, the correct response from a method could be to throw an exception

• We cannot ”capture” this behavior with an assert statement

• Instead, a special ”annotation” is used on the test method

• (expected = name of exception class)

RHS – SOC 28


@Test (expected = IllegalArgumentException.class)

// Test fails if the specified exception is NOT

// thrown

public void testNegativeDeposit()

{

// Note that test as such always succeeds

theAccount.deposit(-100);

}

RHS – SOC 29


• Also, a usual test cannot ”capture” if a method call takes too long (as defined by us) to complete

• This is also handled using a special anno-tation to the test method(timeout = M)

• Method must complete within M milli-seconds, otherwise it has failed

RHS – SOC 30


@Test (timeout = 100)

// Test fails if NOT completed within 100ms

public void testFibonacci()

{

// Calculate the first 10000 Fibonacci numbers

// Note that test as such always succeeds

theFib.calculateFibonacci(10000);

}

RHS – SOC 31

Unit testing considerations

• In the ideal scenario, all units tests should be completely self-contained

• Testing of a particular class should not depend on other classes

• Testing of a particular method should not depend on other methods

• Isolates cause of failed tests

RHS – SOC 32


@test

public void testDeposit()

{

int b = theAcc.getBalance();

theAcc.deposit(500);

int a = theAcc.getBalance();

int diff = a – b;

assertEquals(diff, 500);

}

BankAccount

depositwithdrawgetBalance

RHS – SOC 33


• Suppose now that testDeposit fails

• Which method in BankAccount contains an error…?

• Is it deposit, or getBalance...?

You are wrong!

No, you are wrong!

RHS – SOC 34


@test

public void testCubeVolume()

{

int volume = theCube.getVolume();

int expVolume = theMathLib.calcCube(theCube.getSide());

assertEquals(volume, expVolume);

}

Cube

getSidegetVolume

MathLibrary

calcCube…

RHS – SOC 35


• Suppose now that testCubeVolume fails

• Which class contains an error…?

• Is it Cube or MathLibrary…?

You are wrong, again!

No, you are wrong again!

RHS – SOC 36


• Testing one functionality often assumes that some other functionality already works correctly…

• This is quite hard to avoid in practice

• A rigorous approach is to useso-called test stubs

RHS – SOC 37


• A test stub is a ”simulation” of the behavior of a real class or method

• (Martin Fowler): Test stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test

RHS – SOC 38


• Making a test stub– Write the test, calling any external methods

that are needed– Substitute all calls to external methods with

calls to stub methods (Proxy…?)– Implement the stubs as returning the desired

answer without any calculation

RHS – SOC 39


@test

public void testCubeVolume()

{

int volume = theCube.getVolume();

int expVolume = theMathLibStub.calcCube(theCube.getSide());

assertEquals(volume, expVolume);

}

...

// Code in MathLibraryStub

// Only called with input = 8 in test

public int calcCube(int input)

{

return 512;

}

RHS – SOC 40


• Creating a test using stubs consequently can be done – but is quite labor-intensive

• More pragmatic approach is to use a bottom-up approach– Test basic methods first (methods that do not

call other methods)– When basic methods work, test methods that

only use basic methods– And so on (dependency tree)

RHS – SOC 41

Testing – final remarks

• We can (almost) never expect to create a completely covering test

• Testing is about building confidence in the correctness of the program

• Always a compromise between level of confidence and required effort

RHS – SOC 42


Confidence

Effort

Studentassignment

Commer-cial word processor

Space Shuttle software

RHS – SOC 43


• Further reading:

• JUnit test in NetBeanshttp://www.netbeans.org/kb/docs/java/junit-intro.html

• More about Junit in generalwww.junit.org

• …and the Net contains a lot of material about test in general!

http://www.netbeans.org/kb/docs/java/junit-intro.html

http://www.netbeans.org/kb/docs/java/junit-intro.html

http://www.junit.org/

RHS – SOC 44

Exercises

• Download the NetBeans project IntegerStack from the website (under Classes, week 44)

• The project contains a single class IntegerStack, which has a simple interface with five methods

• Implement a Unit Test for the IntegerStack class, which tests all five methods. Use a bottom-up approach, so you test the simplest methods first (you do not need to use stubs)

• Use the test to discover any errors in the implementation of the IntegerStack class – if you find any, fix them!

• Keep fixing errors until all test cases are green (i.e passed)

testing. rhs – soc 2 testing so far, we have not really created tests in a systematic manner...

Documents

unit test class

test class methods

test packagesunit testing

thorough test

test classfor

xthe test class

generated test code

impossiblenumber of