Enhanced Test Case Design Mechanism for Regression & Impact Testing

Himanshu Joshi and Ravi Teja SurapaneniDept. of Computer Science and Engineering,

JNTUH-CEH,Hyderabad, India

Abstract— Test Case Design and Specifications are mostly written by teams in descriptive manner. Although teams put in their best efforts to write test cases that cover impacted requirements and regression testing scenarios, creating set of all-inclusive test cases is not possible. Also, it becomes difficult for one person to understand & execute test cases authored by another person. This paper examines existing test case design mechanism and proposes a new technique which overcomes the short falls of the existing method and utilizes Test Matrix method for automation.

Keywords— Testing; Test Case; Design; Regression; Automation; Test Matrix

I. INTRODUCTION

In many projects, applications or requirements are validated using Test Case Documents. These test cases are descriptive in nature. The author considers all the possible scenarios for a given requirement and writes down the test cases in his own words. As timeline of project progresses, the requirements change and the document needs to be updated accordingly. Given the fact that different individuals use this document it sometimes becomes difficult for one individual to understand the sentences mentioned in the document by other authors and modify the same. Also, the method does not allow prioritization of test cases. Moreover, since one test case does not map to others, regression testing or testing of impacted requirements is not done effectively.

This paper proposes a new idea to resolve the above problems by using Test Matrix Approach.

Here each field (objects) is mapped against input & output impacted fields. The test cases generated by this approach are more readable and helps realization of related/impacted requirements. This method also uses simple automation techniques to perform regression testing.

II. EXISTING TEST CASE DESIGN MECHANISM

A. Method & TemplateTest Case Design specifies the test case

procedure or input & expected outputs for a given requirement[1]. Each test case is given a unique test case tag or ID. Teams make use of a word document or a spreadsheet for writing & maintaining the test cases. Figure 1 presents sample template of the test case document. In order to know the impacted requirements one has to refer the Bi-Directional Traceability Matrix or mention multiple Requirement Tags for a given test case.

TC_1 FR_1addNumbers - Enter inputs a & b

Output should be 'c' which is equal to 'a+b'

TC_2 FR_2squareSumOfNumbers - Enter input c

Output should be 'd' which is equal to c2

Fig. 1. Sample Test Case Document Template.

When the requirements are too many writing of these test cases becomes very tedious for the authors. Also, the author has to spend a lot of time to analyze the requirements and their impact on each other to write effective test cases.

B. Problems with existing techniqueThe descriptive test case design method has

several disadvantages which are mentioned below.

Author uses his own language which causes difficulty in Reading & Understanding for others

Difficult to Maintain, Update or Manage

Prioritizing of test cases does not happen

Impacted/Related requirements are not captured

Time consuming

III. PROPOSED TEST CASE DESIGN MECHANISM

To solve the problems caused due to descriptive method as mentioned above, the authors propose using Test Matrix approach for writing Test Cases.

A. Method & TemplateIn the proposed technique each test case shall

contain the below list of items. The template is displayed in Figure 2.

Test ID/Tag

Requirement/Function Tag

Impacted/Related Requirement/Function Tag

Input (fields)

Output (fields)

Test Case ID Requirement Tag Impacted Requirement Tag

Procedure Input Output Priority

TC_1 FR_1 FR_2 addNumbers a,b c

TC_2 FR_2 None squareSumOfNumbers c c2

Fig. 2. Proposed Test Case Document Template.

By making use of the above template, impacted requirements cannot be missed and regression testing can be effectively performed. For example, from Figure 2, output of TC_1 impacts the input of TC_2. Hence, every time ‘addNumbers’ procedure is changed, ‘squareSumOfNumbers’ should also be tested. The priority can be set to High, Medium or Low so that the tester can concentrate on major functionality. The priority for a given test case is determined based on the priority of requirements.

To avoid irksome job of finding out the impacted requirements and writing these test cases by hand, the authors propose using Automation and Test Matrix technique. The automation generates test case file which can be readily used for testing.

B. Automation & Test Matrix TechniqueProposed below are the steps to generate

regression test cases for changes in an application.

Test Matrices will generate combinations of test cases which would make sure that no scenario is missed out.

Automation scripts will be generated for the test cases.

The script will execute high priority test cases first and later on medium & low priority ones.

While running these test cases the script will automatically detect related/impacted requirements and run them too.

The results shall be placed in a file and the tester can utilize the report.

C. Benefits in adopting proposed approach Improves readability and understanding of

test cases

Errors are reduced to great extent due to automation

Increases productivity

Easy to maintain

Improves Quality of a product by covering all regression test cases

Confidence of stakeholders and customer increases as defects reduce

IV. RESULTS & FUTURE STUDY

In any given application, it is critical for the test cases to cover all the flows or scenarios. The Test Matrix method would identify all the combinations of inputs and outputs and the automation makes sure that impacted requirements are tested.

Automating the test cases for regression need not involve usage of complex tools. The required steps can be formulated in spreadsheets or Excel sheets using macros.

A simulation of Testing for was done using the exiting and proposed techniques. Below are the features which were evaluated.

Productivity

Decrease in defects leakage

Ease of Use & Maintenance

Reusability

Ease of Audit

Test Case prioritization

For the various parameters mentioned above, the proposed technique outperformed the existing method of writing descriptive test case in terms of rank. The results can be found in tabular format in Figure 3.

Test Case DesignTechnique Feature

Existing Method

Proposed Method

Productivity 2 1Decrease in Defects leakage 2 1Ease of Use & Maintenance 2 1Reusability 2 1Test Case Prioritization 2 1Ease of Audit 2 1Ease of Use & Maintenance 2 1

Fig. 3. Evaluation of Proposed Test Case Design method against existing one

This approach will return good results in non-UI based applications. But for UI based applications, the method should be modified. The authors will continue to work along these lines and bring forth a method for generating test cases in UI based applications.

V. CONCLUSION

In this paper, the authors have analyzed the existing widely used test case design technique and proposed a new mechanism for generating test cases. The proposed approach is easier to follow by the testers. It aims at saving lot of efforts spent on impact analysis and test cases design. The new mechanism also prioritizes the test cases to ensure that important ones are not missed out at any cost. Also, simple automation of identifying regression test cases ensures the coverage of all workflows. Hence, it improves the

quality of the product and confidence of customers and stakeholders.

ACKNOWLEDGMENT

The authors would also like to thank all the near ones who provided support in completing this paper.

REFERENCES

[1] Sample Test Case Template with Examples, http://www.softwaretestinghelp.com/test-case-template-examples/, 2014

[2] Himanshu Joshi, Ravi Leelu Chowdary and Hearsh Varma, "Test Case Design for Critical Systems using Test Matrix and Truth Table", International Journal of Computer Sciences and Engineering, Vol. 2, Issue-04, pp. 12-15, April 2014.