innovative test automation solution

High Reuse, Low Maintenance:A Practical Approach to Test Automation

Alan White

The Nielsen CompanyAlan.White@Nielsen.com

Learning Objectives; You will…

• understand the concepts behind one of the more successful test automation frameworks to date,

• find out which widely used product can be used to quickly produce a visually cohesive, wizard-based GUI tool for creating repeatable test cases, and

• learn how to quickly and intuitively generate and store automated test cases with relative ease while reducing errors – no programming needed!

Problems with Traditional Automation

• Maintenance Intensive• Fragile in Nature• Programming Experience Needed• Static Data• Tightly Coupled to AUT• Minimal Reuse Capability• Unnoticeable Productivity Gains

Solution

• Intuitive• Structured• Maintainable• Role Based• Dynamic Data• Reusable Components

Goals

• Loose Coupling– Application Under Test– Test Automation Tools– Development Platform

• Self Documenting Test Automation Scripts– Scripts look like manual scripts– Detailed account of AUT interactions

• Abstract Complexity from User Community– No programming knowledge required– Focus on building Test Assets

Framework Overview

• GUI Maps– Screens & Components– Controls & Actions

• Test Assets– Test Steps– Test Cases– Test Suites

• AUT Invocation Code (Engine)– Main Loop– Function Libraries (for AUT platform)

Role Based Framework

• Test Engineer (Programmer)– GUI Maps

– AUT Invocation Code

• User Community (SQA, UAT, BA)– Test Script Maintenance

– Test Execution and Results Analysis

• Test Asset Librarian– Naming Conventions

– Other Standards

GUI Maps

• Translate the technical names of screens and components to more meaningful names

Test Steps

• Are the building blocks for Test Cases • Link components to screens and actions to controls• Interact with the AUT to perform specific functions• Should be broken down by their function:

– Navigation– Changing Data– Verifying Data

• Are Highly Reusable Test Assets!

Test Step Example

• Screens and Components are derived from the GUI map• Controls and Actions are mapped to Screens and Components• Parameters store default values that may be overridden

Test Cases

• Are sequential lists of Test Steps that perform specific test operations beginning and ending at the AUT central point

• Allow for overriding existing values to parameters declared in Test Steps

• Resemble manual test cases & narratives

Test Case Example

Test Suites

• Are the highest level of abstraction that specify which test cases are to be executed in the order they should be executed

• Different test suites are defined to operate on different modules or components of an application

Test Suite Example

Hierarchy of Assets

AUT Invocation Code

• Can be developed using commercial test automation tools such as Rational Robot and Mercury QTP or open source tools like WATIR

• Uses Simple Decision Structures• Portable/Maintainable/Reusable• Versatile – Can interact with the AUT screen and

components in the following ways:– Input or Select Values – Verify Values and Properties– Verify Existence

Main Loop• Loops through each instruction (data row)

– Each instruction has its own control

• Branches each control for processing• Reports status to the log

Function Libraries• Contain the functions created for each control• Branch actions in each control function• Can be reused by other applications

Putting the pieces together

Main Loop Control Library

Test Steps/Cases

Test Execution Log

1

23

4

AUT

Microsoft Access Works Well for Building the Front End, Test Asset Creation Tool!

• VBA and SQL• Tables/Queries• Forms/Reports• Data Pages• Macros• Code Modules• Security

Note: More Advanced Deployments might consider developing in an enterprise SDK (e.g. VB, C#, Java).

Getting Started!

How to GUI Map

How to Create a Test Step

How to Create a Test Case

How to Create a Test Case

How to Create a Test Suite

Costs

• Build the Test Asset Creation Tool• GUI Map the Applications Under Test• Build the AUT Invocation Code (Engine)• Train the User Community

Benefits

• Reduced and Known Test Execution Time• Greater Test Coverage• Long Term Cost Savings• Can Run Scheduled and Unattended• Lower Maintenance Effort• Quicker Response to Application Changes

Deployments

• Application Under Test Platforms– PowerBuilder– Java– HTML

• User Communities– Development– SQA– UAT

• Types of Testing– Unit– Smoke– Regression– User Acceptance

Division of Effort

• Dedicated Test Engineer– Ideally 100% Effort Initially– Construction of Framework (Engine)– Construction of Test Asset Creation Tool– Build and Maintain the GUI Maps

• User Community– Create and Maintain Test Scripts– Analyze Test Execution Results– Communicate AUT Changes

Questions

• Please feel free to ask your questions or make comments.

• Thank You!

Further Reading• Nagle, C. “Test Automation Frameworks” White Paper

http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm

• Hayes, L. “Implementing a Test Automation Framework” Tutorial presented at STAR EAST 2005 Conference, Orlando, FL, May 16, 2005.

• Automated Testing Specialists, Methodologies for Automated Testinghttp://www.sqa-test.com/method.html

• Zambelich, K. Totally Data-Driven Automated Testing 1998http://www.sqa-test.com/w_paper1.html

• LogiGear Whitepaper Series, Achieving the Full Potential of Test Automation October 25, 2004

• Mosley, D. & Posey, B. Just Enough Software Test Automation New Jersey: Prentice Hall PTR, 2002.

• Buwalda, H. & Janssen D. Integrated Test Design and Automation Great Britain: Addison-Wesley, 2002