beyond the gui
TRANSCRIPT
-
7/28/2019 Beyond the GUI
1/31
Testing Beyond the GUI with Rational
Visual Test
Daniel Rybowski
IT ManagerHitext
-
7/28/2019 Beyond the GUI
2/31
Based on real world experiences...
Explain the limits of the Capture/Playback test
automation
Examine an alternative approach to GUI test
automation
Outline the implementation of a test
automation tool with Rational Visual Test
Objectives of this session
-
7/28/2019 Beyond the GUI
3/31
Presentation outline (1/4)
The context
An application under test: architecture
and technology
A test strategy: scope of the tests, test
methodology
-
7/28/2019 Beyond the GUI
4/31
Presentation outline (2/4)
Initial approach: test automation withSQA Robot
Selecting the test tool
Our experience with Capture/Playback
Why we failed
-
7/28/2019 Beyond the GUI
5/31
Presentation outline (3/4)
A new approach: the Programmed model
What is it?
Comparison with Capture/Playback
A new test tool: Rational Visual Test
-
7/28/2019 Beyond the GUI
6/31
Presentation outline (4/4)
The Programmed model in practice: the test
driver Pillory
Global design
Using Rational Visual Test features
HiTest: the future of Pillory
-
7/28/2019 Beyond the GUI
7/31
The context (1/5)
An application under test: architecture and
technology
Client/Server software
Client-side applications for Windows 3.1
and 95
Multi-lingual Client applications
-
7/28/2019 Beyond the GUI
8/31
The context (2/5)
An application under test: architecture and
technology
Server-side applications for Windows NT
Client and Server databases
GUI based on client-customizable viewsand custom controls
-
7/28/2019 Beyond the GUI
9/31
The context (3/5)
A test strategy: scope of the tests, test
methodology
Scope of the tests: conformity of the
software with its specification
No performance, usability,... testing
Behavioral (black box) tests only
-
7/28/2019 Beyond the GUI
10/31
The context (4/5)
A test strategy: scope of the tests, test
methodology
Test design based on the specified
business rules
RAD software development
methodology with incremental builds
-
7/28/2019 Beyond the GUI
11/31
The context (5/5)
A test strategy: scope of the tests, test
methodology
Decision: the tests must be automated
-
7/28/2019 Beyond the GUI
12/31
Initial approach: test automation with SQA Robot (1/4)
Selecting the test tool
The tests cannot be automated in
advance:
custom controls
specifications subject to change
Decision: use a Capture/PlaybackCAST tool
-
7/28/2019 Beyond the GUI
13/31
Initial approach: test automation with SQA Robot (2/4)
Our experience with Capture/Playback
Creation of the test cases
Verification of the test results
Maintenance of the test cases
Verification of the fixed defects
-
7/28/2019 Beyond the GUI
14/31
Initial approach: test automation with SQA Robot (3/4)
Why we failed
Functionality changes of the application
under test
GUI changes of the application under
test
Custom controls handling
-
7/28/2019 Beyond the GUI
15/31
Initial approach: test automation with SQA Robot (4/4)
Why we failed
Sensitivity to environmental changes
Sensitivity to change of locale
Atomicity of the recorded test cases
Test engineers need a wide expertise
-
7/28/2019 Beyond the GUI
16/31
A new approach: the Programmed model (1/9)
What is it?
A new model of GUI test automation
Separates the test case logic from the
interaction with the application
Test Designer and Test AutomationEngineer are distinct roles
-
7/28/2019 Beyond the GUI
17/31
A new approach: the Programmed model (2/9)
What is it?
Three-tier model:
test cases description in high level
script language
generic test driver, interpreting the
test scripts
low-level primitives to interact with aspecific application under test
-
7/28/2019 Beyond the GUI
18/31
A new approach: the Programmed model (3/9)
What is it?
Test scripts in high level language
GUI access primitives
Test driver
-
7/28/2019 Beyond the GUI
19/31
A new approach: the Programmed model (4/9)
What is it?
The driver collects and logs test results
The test scripts are independent of any
GUI changes
The GUI access primitives areindependent of any functional changes
-
7/28/2019 Beyond the GUI
20/31
A new approach: the Programmed model (5/9)
Comparison with Capture/Playback
The Programmed model has up-front
costs:
The test driver (once)
The primitives (once per application
under test)
-
7/28/2019 Beyond the GUI
21/31
A new approach: the Programmed model (6/9)
Comparison with Capture/Playback
The Capture/Playback model has higher
implementation costs
The Capture/Playback model has higher
maintenance costs
-
7/28/2019 Beyond the GUI
22/31
A new approach: the Programmed model (7/9)
Comparison with Capture/Playback
The Capture/Playback model can lead to
failure:
for functional tests
if the application is big
if the functionality is complex
if the development cycle isincremental
-
7/28/2019 Beyond the GUI
23/31
A new approach: the Programmed model (8/9)
Comparison with Capture/Playback
The Capture/Playback model can lead to
failure:
if the functionality changes often
if the GUI changes often
if there are custom controls
if the application is localized
-
7/28/2019 Beyond the GUI
24/31
A new approach: the Programmed model (9/9)
A new test tool: Rational Visual Test
Strong programming environment
Powerful language primitives
Interfaces with a version control system
Inexpensive
The Programmed model in practice: the test driver
-
7/28/2019 Beyond the GUI
25/31
The Programmed model in practice: the test driver
Pillory (1/5)
Global design
Master/Slaves architecture to allow for
Client/Server testing
Single test script for all test stations, run
by the Master
Test scripts in Microsoft Excel sheets
The Programmed model in practice: the test driver
-
7/28/2019 Beyond the GUI
26/31
The Programmed model in practice: the test driver
Pillory (2/5)
Global design
Low-level primitives programmed in
Visual Test
Handles test suites as trees of test
cases
The Programmed model in practice: the test driver
-
7/28/2019 Beyond the GUI
27/31
The Programmed model in practice: the test driver
Pillory (3/5)
Using Rational Visual Test features
Master/Slaves dialog with the Talk
primitives
Access to the Excel sheets with ActiveX
automation and the Ole primitives
Interaction with the Win32 API
The Programmed model in practice: the test driver
-
7/28/2019 Beyond the GUI
28/31
The Programmed model in practice: the test driver
Pillory (4/5)
HiTest: the future of Pillory
Cleanroom implementation of Pillory in
Java
Uses Rational Visual Test DLL
Processes asynchronous events
The Programmed model in practice: the test driver
-
7/28/2019 Beyond the GUI
29/31
The Programmed model in practice: the test driver
Pillory (5/5)
HiTest: the future of Pillory
Testing of Web pages
Testing of localized software
Public release
-
7/28/2019 Beyond the GUI
30/31
Thank you!
-
7/28/2019 Beyond the GUI
31/31
Any questions?