white box testing in practice - qai global...
TRANSCRIPT
XPANXION INTERNATIONAL PVT. LTD.
Server Space, A. G. Technology Park, Off ITI Road, Aundh, Pune – 411007
White Box Testing In Practice
STC 2013 Conference
By -
Smita More (email : [email protected] )
Chaitanya Dharmadhikari (email : [email protected] )
White Box Testing In Practice Page | 2
ABSTRACT
White box testing encompasses several testing techniques used to evaluate the logical
functionality of an application, block of code or specific software package. It helps to
improve the quality of the software application by identifying a majority of bugs early in the
software development lifecycle. This in turn reduces the cost of fixing them at later stages.
This paper introduces white box testing along with the details of how it is implemented in
real world projects. This paper will elaborate the challenges in white box integration testing
and BPM unit testing from the practitioner‟s point of view and the approach used to
overcome them.
The paper attempts to describe:
1. Different methods that can be used to add unit and integration test cases (using the
Spring Framework (open source application framework), Maven and JUnit).
2. The guidelines that should be followed while writing unit test cases.
3. Different approaches that aid in writing unit test cases. (We have used the Mockito
framework with JUnit and its best practices as an example).
4. White box testing of BPM processes: With the increase in use of BPM software in
nearly every organization, this paper tries to explain how white-box testing can be
applied to test BPM processes as well.
Our approach has helped us to reduce manual testing efforts and improve the overall
productivity of automating unit and integration test cases.
White Box Testing
Let us first understand what white box testing is before focusing on its numerous benefits.
White box testing is testing based on an analysis of the internal structure of the component
or system. It means testing the minute internal workings of the component or system under
test. For example, opening the engine block of a car to check whether the pistons are
working properly. An example in terms of the software industry is writing tests in order to
execute every statement inside a particular method of a class.
We will explain the following approaches to white box testing along with the benefits of each
in this paper -
1. Unit testing – Testing of individual methods of a given class.
2. Integration testing – Testing end to end flows through multiple classes.
3. Testing BPM processes – Testing of high level business flows.
White Box Testing In Practice Page | 3
1. Unit Testing
Unit testing in its simplest form is testing each part of the system individually. E.g. when we
test the indicators of a vehicle, we aren‟t bothered whether the vehicle has two wheels or
four. We are focused on whether the indicators blink their intensity etc. Similarly in
software, we test the functionality of an individual class by means of a unit test.
A unit test is a test related to a single functionality of a method of a class. The purpose of
unit tests is to verify that the specific unit of code works as expected. The idea behind unit
testing is that we want to test our bit of code regardless of its dependencies. A dependency
is when one class in an application depends upon another in order to perform its intended
function.
A unit test is written using fake replacements of the required dependencies. Mocks are a
type of such fake replacements. Mocking by definition is creating a copy of a real object. In
case of Figure 1.1 below, class A has dependency on classes B & C. So in order to test class
A we provide fake functionalities by mocking classes B & C.
Figure 1.1
Some easily available tools or frameworks like Mockito or PowerMock enable us to create
mocks in a simple and intuitive way, while at the same time providing great control of the
whole process. Mockito artifacts are available in the Maven Central Repository (MCR).
Maven is a commonly used build tool (For more information, please see
http://maven.apache.org/). The easiest way to make MCR available in your project is to put
the following configuration in your dependency manager:
Maven:
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>1.9.0</version>
<scope>test</scope>
</dependency>
Using Mockito, we can focus more on the class under test than the dependencies that are
required to carry out the test. This reduces testing efforts as we need not spend time in
Class A
Class B Mock
Class C Mock
White Box Testing In Practice Page | 4
looking for suitable dependent objects that match our requirements. We can just create
temporary objects of our own as and when required.
Let us now understand how to use mocking in our projects for different types of scenarios.
The project code base is in java and we have used the JUnit test framework for our
examples.
a. Mocking methods that return a value
Consider the following line of code -
VendorOrder vendorOrder = VendorOrderService.findById(vendorOrderId);
Suppose we have a java class named “AvmServiceImpl‟ under test that uses an
object of class “VendorOrder”. Now if this VendorOrder object is retrieved from a
method of a service say “VendorOrderService”, we cannot spend efforts on getting a
real value from this new service. Our focus is on the class under test and not its
dependencies.
We use a mock in this case. One of the basic functions of mocking frameworks is an
ability to return a given value when a specific method is called. It can be done using
Mockito.when() in conjunction with thenReturn(). This process of defining how a
given mock method should behave is called stubbing.
Here, VendorOrderService is an external service used in the class under test. Hence
we need to stub the returned value of the method findById() of VendorOrderService
and return a fake replacement of the real value. This can be stubbed as below:
when(VendorOrderService.findById(vendorOrderId)).thenReturn(vendorOrder);
where this “vendorOrder” object can be created in our JUnit test class.
If you want a method to throw an exception, above method can be stubbed to throw
an exception.
when(VendorOrderService.findById(vendorOrderId)).thenThrow(Exception.class);
Mockito provides a family of functions for requesting such specific behaviors. Few of
them which are used often are
Method Description
thenReturn(T valueToBeReturned) returns given value
thenThrow(Throwable toBeThrown) throws given exception
White Box Testing In Practice Page | 5
b. Mocking void methods
As we have seen above, the stubbed method is passed as a parameter to a method
that returns some value. This obviously means that you cannot use this construct for
void methods. Instead, you should use a doSomehing ()..when () construct. See a
few methods below:
Method Description
doThrow(Throwable toBeThrown) Throws given exception
doNothing() Does nothing
c. Argument matching
Mockito, by default, compares arguments using equals methods. Sometimes it‟s
convenient to know exactly what parameter the method will be called with. We can
match exact object values, the types of objects, the types of object lists and so on.
An example of Argument matching is as below:
when(postalAddressService.createPostalAddress(eq(address1), eq(address2),
anyString(), anyString(), anyString(), anyBoolean(), Matchers.anyList(),
Matchers.anyListOf(MasterZip.class))).thenReturn(new PostalAddress());
d. Verifying Behavior
Once created, a mock remembers all operations performed on it. These operations
can easily be verified using Mockito.verify(T mock) on the mocked method. By
default, Mockito checks if a given method (with given arguments) was called exactly
the required number of times. Mockito provides a number of verification modes. It is
also possible to create a custom verification mode.
Name Verifying method was...
times(int wantedNumberOfInvocations) called exactly n times (one by default)
never() never called
atLeastOnce() called at least once
White Box Testing In Practice Page | 6
A Simple Mockito test case example is as below:
Method under test –
JUnit test –
Since Unit tests cover every single statement or condition within a given class method, the
test coverage will always be high. Coverage tools like Clover can be used to calculate the
code coverage of tests that are executed. Using Mockito increases the productivity of white
box testers. We were able to test more than 90% of the source code though the client
expectation was only 70% and that too within the initially estimated time frame. Hence, unit
testing overall gives the client more confidence in the system that is being built.
We will now look at the second approach in white box testing i.e. integration testing.
White Box Testing In Practice Page | 7
2. Integration Testing
Let us use the car example to understand integration tests. Now that we are sure that each
individual part of the car is working as expected, we need to make sure that these parts
function properly when they are combined. We take the car for a test drive and check
whether the engine is running smoothly; the brakes slow the car down; the indicators work
as designed and so on.
Similarly in software, integration tests are used to test the entire end-to-end functionality.
In most projects we need to obtain data, process it and save the updated data to some
database. Using integration tests we need to verify whether the right data is retrieved, the
right processing is done and that correct results are saved in the database. Hence, our
tests too require a database to reflect changes at the different intermediate levels.
A logical representation of a real world project can be as shown in Figure 2.1 below -
Figure 2.1
Consider an example of a Controller under test. A controller interacts with a Service “B”,
which then gives a call to the data layer (i.e. repository) that interacts with the database.
Though the basic structure of an integration test is similar to a unit test, integration tests
require a call to real services and repositories unlike unit tests using Mocks.
These integration tests can be time consuming as they cover multiple class methods and
require the database to be loaded before execution. One significant improvement in this
regard is to load only the tables necessary for execution of the specific tests instead of
loading the entire database each time. We provided only the main tables necessary for
execution in the tests. The DBUnit Framework picks up the dependency tables
automatically. This reduced the manual efforts needed for identifying the dependency tables
from the entire database. Also, loading only selective tables reduced the overall build time
from „40 minutes‟ to „30 minutes‟. One of the approaches used for white box testing
required insertion of data in the xml files. Approach of inserting this data through White Box
Integration Testing was reduced and the client could gain a profit of “90 days”. So just
imagine in terms of cost gained by the client. I am sure every client would like to adopt
such approach for their projects.
White Box Testing In Practice Page | 8
Integration tests check the complete flow through different modules and ensure that the
system as a whole works fine. However, we cannot say that all conditions within a given
class method will be covered. Hence, integration test coverage will rarely be as good as unit
test coverage.
The next challenging area in White Box Testing is testing BPM Processes.
3. Testing BPM Processes
Many Organizations implement BPM (Business Process Management) solutions. BPM
processes can range from simple (having a single flow with a few tasks in it) to very
complex business flows. Those who need to understand the BPM processes in detail can
refer to the References section of this White Paper.
With the complexity of the business flows, there lies a challenge in understanding the flows
and testing them. Like Unit testing, there are different tools available in the market that can
create a unit test for the BPM process. For example, there is a tool called „Activiti‟ by
Alfresco. This tool can be added as a plugin to the Eclipse editor. It provides a designer view
where one can just right click on the process name in their project and add a unit test. This
can be done when you just need to add a unit test for a simple BPM process (no sub-
processes in it).
We faced a challenge in testing BPM processes that had different sub-processes in it. Here
the challenge is to test one process at a time. Before we go into any details about the
approach, we would like to first explain the context in our project. Unit testing of more than
90% of the Java classes was completed and we needed an optimum approach to test the
flow of each process where we verified invocation of the tasks associated with those
processes. There is no defined approach or any standard guideline that can help you in
adding test cases for BPM. We came up with this custom approach to test each of the BPM
processes. We would rather call this a hybrid approach because the approach involves use
of both unit and integration testing. How it is both unit and integration is explained in detail
below.
Let us take a live example used in the project. At this point, do not bother about what each
of the tasks and processes below do. Our intent now is to just understand the high level
flow and the approach to test it.
White Box Testing In Practice Page | 9
Figure 3.1
Above diagram represents one of the sub processes named „PfcSubProcess‟ in a main
process called „Vendor Order Main Process‟. The Activiti tool provides an interface to view
the properties associated with each construct (this can be viewed in any BPM tool). Let us
now go through each of the construct properties and understand the flow.
Figure 3.2
As seen in the properties, this above task does not have any associated Java class. Any
script can be run through this task if required. In this case it just prints a message.
Figure 3.3
After the completion of the Script task, the PFC Notification Service Task would be invoked.
White Box Testing In Practice Page | 10
Figure 3.4
Figure 3.5
After the PFC Notification Service Task, above two user tasks are „Corrections Required‟ and
„Follow up with Vendor for Fulfillment Corrections‟. Both tasks will be run in parallel.
White Box Testing In Practice Page | 11
Listeners would be called after completion of each of the tasks. In a BPM test, user tasks
need to be completed through code, as these tasks are carried out by a user manually
through user interface. We will look into more detail below.
As mentioned earlier, the two processes are sub processes of the main process. Our
objective is to test these sub processes. Each sub process may require input from another
process and give some output at the end which again can be an input to another process.
Let us now understand the technical aspect required for testing the BPM process.
It is a best practice to have a base class with all common methods required for testing
processes. This Base class should include:
a. Context Configuration required for your project. These xml files should include
the mock beans for the java classes used in the Listeners, actual beans for the
service/user tasks in your BPM process. For example above, we will need to add a
real object for PFC Notification Service Task and mock objects for the service calls
within the PFC Notification Service Task.
b. Methods to start the job executor and stop job executor. The JobExecutor
is an interface provided by Activiti that is used to start and stop the BPM process.
c. Method to get the sub process instance id. This can be done by querying
database using the main process instance ID.
The JUnit class for the BPM process should include:
a. Autowired beans. (Injection of dependent objects or beans)
b. Application configuration. This requires the name and version of the process
White Box Testing In Practice Page | 12
c. A variable map based on the inputs required for the process. This can be
analyzed based on the flow of the BPM process.
d. Stubbed calls used in the listeners to return expected object. You can refer to
Unit testing section.
e. Execute the process using RuntimeService and Repository Service. (You can find
details of the Core interfaces in the links provided in References Section)
ProcessInstance instance = runtimeService.startProcessInstanceById(processDefinitionId,
businessKey, paramMap);
ProcessDefinition definition =
repositoryService.createProcessDefinitionQuery().processDefinitionId(processDefinitionId)
.singleResult();
As mentioned above, if the BPM process includes a user task, this task needs to be
completed using code. Please see the example below.
Next important point to conclude a test case is assertion. Once we execute the process
using runtimeService, we should check if the expected tasks are created. In this example,
we should assert if the two user tasks are created. Please see code snippet above.
White Box Testing In Practice Page | 13
The advantage of testing the BPM process is that you can uncover the gaps in the flow even
before the application goes into the Black box testing phase.
In conclusion we would like to summarize the overall benefits of white box testing…
Benefits of White Box Testing –
Unit tests clean up the source code. They ensure that class methods are performing exactly
as required and that no extra functions are carried out; unnecessary conditions are
removed; looping structures are simplified and that overall proper coding standards are
followed. This improves overall code quality and performance of the system. Statistics have
shown that finding and correcting defects at earlier stages of development is more cost
effective. Figures from “Applied Software Measurement”, (Capers Jones, McGraw-Hill 1991),
for the time taken to prepare tests, execute tests and fix defects show that unit testing is
about twice as cost effective as integration testing and more than three times as cost
effective as system testing.
Effectiveness of Early Testing
Integration tests ensure that all modules are interacting perfectly and that there are no
breakages. The database functionality is also tested thoroughly.
BPM tests ensure that process flows have the best possible design and that they are
efficient.
Since overall quality of the entire system improves by white box testing, clients are satisfied
with the quality that they receive at the end and customer satisfaction leads to customer
retention. Customer retention increases the overall brand value.
White Box Testing In Practice Page | 14
References
Mockito FrameWork:http://docs.mockito.googlecode.com/hg/org/mockito/Mockito.html
BPM Process tool Activiti with details of interfaces required for adding unit test cases
forBPM:http://www.mif.vu.lt/~donatas/PSArchitekturaProjektavimas/Library/BP/2010%20B
PMN%202.0%20whats%20in%20it%20for%20developers%20%28Activiti%29.pdf
Foundations of Software Testing
Authors’ biography
Smita More, having 6+ years of experience in IT industry. During this tenure I have mostly
worked in Media Data Advertising and Broadcasting Domain, Financial and Mortgage
Services. I am working with Xpanxion International Private Ltd since last 4 years. I am
working as a QA Lead and handling the team size of 5. In our current project we are
working on White Box Testing using technologies Spring Framework, Spring Integration,
JUnit and Maven. I also have experience in Automation using Selenium Web Driver, Test
Complete, Sikuli, Cucumber, ETL Testing and Web Services Testing. Technical Languages
that I am familiar with are Core Java, SQL, Python, Shell scripting. Databases used are SQL
server, My SQL, Oracle, PostGreSql.
Chaitanya Dharmadhikari, having 2 years of experience in the IT industry. I have worked
in Hotel, Real Estate and Weather Forecasting domains. I am working with Xpanxion, India
right from my internship days. I have done both black box testing and white box testing. In
our current project I am working on automation testing for an iOS application. I have
worked on various tools like HP-QC, Jira, XCode and Idea. The technical languages I am
familiar with are Java, UI Automation JavaScript and SQL. The databases used are SQL
server and MySQL.