Testing: Agile & OO

Robert JohnsonMelissa Schwartz

Testing Object-Oriented Software Systems

Harry M. Sneed

Object Oriented Systems:

The advantage is a disadvantage in terms of testing:

• flexibility and usability o  many ways to use and therefore many needed tests

"One can summarize that the greater the variability and flexibility of a piece of software, the more the usage possibilities are, then the more there is to test."

The solution to the challenge?

• Automation

Higher cost:

"Boris Beizer, the dean of Software testing, wrote in an article in the American Programmer, saying that it would cost a lot more to test 00-software than to test conventional procedural software – perhaps 3 or 4 times as much."

Why?

• object-oriented class hierarchy has to be retested for every context in which it is reused

• little differences in the environment can cause big side effects

"James Martin, the guru of data – driven software development pointed out, that reusing classes from class libraries was like eating spaghetti. If you wanted to pull out one strip all of the other strips come along too. They are all intertwined with one another. So it is not possible to select and test individual classes. If you want to reuse and test one, you have to test them all."

Particular drivers of test costs are exactly those features which distinguish object-oriented software from procedural, problem specific software:• encapsulation

• generalization

• association

• polymorphy

Encapsulation:

• In procedural systemso gain access to each and every field in the storage

all kept in a single static storage space fields can be set, altered and checked indiscriminately

• In object-oriented systems o  constructor 

 reason for the emphasis on built-in tests when testing classes - the test has to be part of that particular class under test

Since there is a different test for every class, that increases the test effort

Generalization:

Generalization increases test effort by requiring us to test every context in which a super class can be used. 

• sub classes can be added but we can't be sure that the same super class really fits to them

• the deeper the inheritance tree is, the more branches there are to test

• each additional branch of the tree may have side effects on the other branches, so these too have to be retested

Association:

Association allows for any object to be associated with any other object in the system. 

• accomplished by invoking a foreign method in another class hierarchy 

• each such association has to be tested

• unlimited (spaghetti)

Polymorphy:

Polymorphism makes it possible to select methods for execution at run time. 

• makes the code much more flexible and general, but it is a nightmare for testing (since every possible variation of the software should be tested)

• Testing a polymorphic method invocation means testing every potential target method

Polymorphy:

Four approaches to class testing:

Non modal means testing every single method with a single object instance. 

Uni modal means testing every combination of methods with a single object instance. 

Quasi modal implies testing individual methods with multiple object instances

Modal implies testing every combination of methods with multiple object instances (all possible object states)

• plus conventional challenge of testing every path through a method

• combining paths, with methods, with states leads to a combinational explosion of potential test cases

Solution:

automatically generating the test cases and joining these test cases with those derived from the class specification

without the support of automated class test beds there is no way to test complex classes (tools like JUnit, CPPTest and NetUnit can fulfill this need if used correctly)

Integration Testing:

• verticalo testing of inheritance connections

• horizontalo testing associations

Greatest challenge with integration testing is testing the way components interact with each other Possible kinds of component interaction:• they share a database• they send messages and receive responses• they use another components methods

Solution:The integration test of components is based on the interface descriptions. As such it is in effect an interface test. 

The test cases are extracted from the interface specification. • formal interface specification language, which also includes the

value domains of the arguments and result (CORBA-IDL, XML, WSDL or SQL)

The goal is to test all potential combinations of arguments• may be too many arguments to justify the costs• may be necessary to select certain representative combinations. 

The first approach is automatable. The second requires some form of human interaction to select combinations of states which are representative of the others. 

When Agile Meets OO Testing - An Approach to testing OO Agile Projects• Given the nature of Agile, old traditional testing using a detailed

test plan isn't ideal• To adjust to Agile and volatile requirements it becomes necessary

to implement automated testing, throughout the life-cycle of a projecto automated tests allow 'fast' execution with a high return value

The Approach - Informal Reviews

• during development the developers will test their code against the requirements for the iteration/sprint

• before each release the developers and testers meet to verify the build is readyo current sprint feature-set is checked, if there are issues they

must be fixed before the build can be released

The Approach - Test Automation Pyramid

Lower tierMiddle tier

Top tier

Progressing from lower to top tests become more expensive, requiring more time and resources to complete

The Approach - Test Automation PyramidLower Tier

• Lowest tier consists of unit and component testingo when needed this will include using a mock testing framework

so testing can be done without relying upon back-office services

o since these can be written and maintained easily while also being run quickly they have the higher ROI as many tests as possible should be implemented at this tier

o All builds must pass this tier of testing before being released for further testing

The Approach - Test Automation PyramidMiddle Tier

• Middle tier consists of testing high-level business functionality and and back-office services o  this is done using a tool such as JUnit, allowing testing to be

done independently from the presentation layer this is done as it is less expensive to write and maintain

o however, these tend to be slower since they cover a large breath of functionality and accesses back-office services with delays

The Approach - Test Automation PyramidTop Tier

• Top tier consists of testing the presentation layer using a testing framework o this is done using a tool that typically uses low level language

interfaces to trigger GUI interactioinso given the 'brittle and expensive' nature of GUI tests there

should only be a small investment in top tier testing

The Approach - Load/Stress Testing

• For systems with back-office services, testing of these services should be done directly, without integrating other development layerso tools such a JMeter should be used

opensource and can be extended as neccessary

The Approach - Manual Testing

• Agile projects will have manual testing as well buto it must be minimized, allocating as few resources as possibleo it is expensive and test cases don't carry on wello pushed as far forward a possible in the sprints

pushed to more stable builds of the project 

The Concept of "Good Enough Testing"

• Since Agile iterations are short and rework is the norm, insuring that only just enough testing is done to make the decision to continue or stop is keyo is the quality assessment sufficiento are the costs reasonableo can timely/wise decisions be made

Agile Testing Wrap-up

• Each iteration/sprint is tested• Automated testing is a must• Good Enough Testing is paramount

Conclusion:Testing OOP is problem at the unit and integration testing levels. 

In testing object – oriented systems, the test is moved up to a higher level of abstraction, where test automation is absolutely necessary.

• increased testing effort due to the increase in possible usages and to the greater number of potential interactions

•  the increased flexibility and usability of the software under test makes it all the more important to automate - since only an automated test can cover the code in combination 

•  by automating the test, emphasis is shifted from the manual creation of test cases to the manual creation of test scripts