8/9/2019 Testing 2007

1/8

CO531: Software Engineering Practice:

Testing

or some of the things you should

know about testing but never knew

you had to ask!

Books

Amazon returns 400+ hits for softwaretesting in the title

Recommended:How to Break SoftwarebyJames A. Whittaker, Addison Wesley, 2003.Price: 25.19 (from Amazon)

Google returns about 175,000,000 hits to asearch on software testing

Truth vs Anecdotes?

Testing

Why do we do it?

Because we aint perfect

When do we do it? All the time

How do we do it?

Systematically and repeatedly

When do we stop doing it?

When the software isnt used any longer

Simple Example

CO522 Example class

p(x) = 4096x6 36864x5 + 129280x4

222720x3 + 194224x2 78096x + 10395

Did anyone test their calculation method?

How did/do you test this?

Balance?

8/9/2019 Testing 2007

2/8

Exhaustive testing

Exhaustively test a method that takes two

integer parameters. Assume

32-bit integers

runs a test every 50 nanoseconds (10-9 secs)

How long will this take?

Exhaustive testing .

is generally not an option.

General Introduction

More next term

What is the point of testing?

XP is very hot on testing

What does testing actually tell you?

What cant testing tell you?

Confidence building but dont forget

that Oh fcuk moment

Dijkstra says

Program testing can be used to show the presenceof bugs, but never to show their absence!

You must not give the world what it asks for, butwhat it needs.

We must not put mistakes into programs becauseof sloppyness, we have to do it systematically andwith care.

Object-oriented programming is an exceptionallybad idea which could only have originated inCalifornia.

8/9/2019 Testing 2007

3/8

Errors --Nomenclature

looking for bugs, errors, defects, faults,

failures, features, showstoppers

lost marks, wrong answers, dead people,

crashed aircraft, no passports, blue screens,

any of the above (Microsoft)

Debugging fixing it safely!

Removal cost increases with time

Why/How Do Errors/Bugs Occur

Designed/written/tested/used by

fallible/malicious people.

You made a silly error a simple oversight

Youre being pushed to complete a project

Unforeseeable circumstances occur

Difficult/impossible to test some

circumstances for real

Types of Testing

Just talking about testing code

Unit testing vs System testing

White/Glass box, grey box, black box

Iterative testing vs Big bang testing

8/9/2019 Testing 2007

4/8

Story

Understand

Add one failing test

Add production

code for that test

Run ALL tests

Rework OK Story

Complete

Next

Story

Never write a line of

functional code without

a broken test. Kent Beck

The Test First (TDD) Scheme

Why Test-First

Feedback: instant on code changes

Task orientation: focuses on problem

decomposition, maintains focus, provides

steady and measurable progress

Quality: built-in and maintained via testing

Low-level design: context for decisions

(classes/methods, names, interfaces, )

Advocates and Sceptics

Advocates: increased productivity andbetter quality

Sceptics: Hard to learn, counter-productive Quality: Interleaving tests with coding

Productivity: Not so obvious

Studies: inconclusive

Other factors: team dynamics, self-esteem,courage

What else is it?

Test first coding is not a testing technique

Ward Cunningham (being provocative!)

Analysis technique

Design technique

and, I believe, its also a testing technique!

8/9/2019 Testing 2007

5/8

Unit Testing

Usually tests just one particular method

Need confidence that low-level routines workbefore adding high level code

Not concerned with formal validation of code

Still need other forms of testing

Wish to avoid the ripple effect

Good unit testing

Better code design

Less time in the debugger

Unit Testing contd

Answers the question Is the code doing exactly what I think it should?

Note that this isnt the question Does the code do what the requirements demand?

You need to ensure your code does what you want ALLTHE TIME Regression tests all tests must pass NOT JUST THE NEW

ONES

Unit testing helps with documentation Tests are visible

Test are up-to-date and correct (because you keep running them!)

Code documentation (on the other hand) tends to drift

Unit Testing

Testing isnt free but ...

Invest up-front to save time later

Typical excuses It takes too long to write tests

It takes too long to run the tests

Its not my job to test my code

I dont know what the code is supposed to do so I canttest it

I feel guilty putting testers and QA staff out of work

Im not allowed to run unit tests on a live system

JUnit

Nice to have help with unit tests

BlueJ provides a neat interface to the JUnit

testing framework. Dont have to write drivers

You have to know the answer to each test

You can build the tests up

You can run all the tests easily (and batches ifyou want to do a bit more work.

8/9/2019 Testing 2007

6/8

Unit Tests vs Function Tests

Unit tests are local

Include integration tests

Written by (and for) programmers

Function testing = higher order testing

Does code do what the user expects?

XP has continual customer involvement

Iterative function testing = minimal surprises

Done by end user (as well as programmers)

Communication

Others: stress, performance, security, installability

Generating Test Cases

Problem dependent -- experience

Some general rules

equivalence classes one example covers many

edge cases what happens at the limits (< |

8/9/2019 Testing 2007

7/8

Basic Guide to Debugging

1. Recognise that there is a defect inyourcode

Check input data (program should already do this?)

Reproducibility

Identify the symptoms

2. Isolate the source of the problem

What part of the system is causing the problem

Check inputs and outputs over smaller parts of the

system

Experience counts here

Basic Guide (contd.)

3. Why has the problem occurred?

Incorrect code may be a long way from the problemsappearance (code distance/time)

E.g., incorrect value generated but problem notdetected until it is used much later

Are there similar sections of code elsewhere?

4. Determine the fix

Sometimes simple one line

Sometimes catastrophic major rewrite

Quick fix may be necessary

Basic Guide (contd.)

5. Fix, TEST, TEST and TEST

Apply the fix

Check it does correct the problem (TEST)

Check nothing else breaks (TEST)

Add new test (which one???) to regression

(and unit ??) tests

Useful things to do

1. Have the correct mind-set Your code probably is broken

2. Add assert/debug statements into code so they can be

activated when necessary3. Use log files

4. Only ever make one substantial source change at a time Helps prevent adding new bugs

5. Back out of changes that dont fix the problem Problem not where you think it is

Modified code never called

Use a source code control system (Subversion/CVS)

8/9/2019 Testing 2007

8/8

Do I Only Add Failing Tests

Never write a line of functional code

without a broken test. Kent Beck.

Doesnt say I cant add tests that dont fail!

Add test if

they show a failure (need to write code)

there is something special about the test

Document all tests

Testing GUIs

Is a form of function testing

XP approach is not as successful (Kent

Beck says so so it must be true)

Its hard to do

Its hard to automate

Screen shots are worth a thousand words.

Whittakers Book

Read his book for a really useful list of

things to try when you perform both unit

and function testing.

Try some of these attacks out on your

project code.