image based testing-indicthreads-q11
DESCRIPTION
Session Presented at 2nd IndicThreads.com Conference On Software Quality held on 25-26 March 2011 in Pune, India WEB: http://Q11.IndicThreads.com ------------ Session Abstract Image-based testing (IBT) is a new form of automated software testing that relies solely on evaluating and\or duplicating the user-experience through the on-screen display and user-input devices. Image-based testing requires the test automation engine be able to acquire artifacts from the on-screen display or otherwise process snapshots of the on-screen display to perform the tasks assigned required for an automated test. It utilizes image recognition to perform mouse clicks and keyboard strokes just as a user would. IBT is technology agnostic because it is only looking at what is on the screen. As a result, the application under test can be written in almost any language, such as Java, .Net, or Flash. Image-Based testing allows us to test virtually anything that can be displayed on the screen. It doesn’t matter what the underlying technology is, if it is visible on the screen then we can interact with it. Tool manufacturers can hardly keep pace with the needs of cutting edge application testers. If you are tasked with testing newer or novel technologies then it is usually difficult to find a tool that can work in that environment early in the development and testing lifecycle. When the tools you have can’t support the technologies you need to test you need to turn to something else. This is a good time to consider Image-Based testing (IBT). This session will look at: What exactly IS Image-Based Testing? Thinking and Testing Differently with IBT. Developing IBT Tests and App Maps. SAFS Image-Based Recognition Syntax. Dealing with Variations in the IBT Environment. Dealing with NLS Testing in an IBT Environment.TRANSCRIPT
Image Based Testing- application technology
independent automation
Girish KolapkarSAS R&D
(India)
Agenda:• What exactly IS Image-Based Testing (IBT)?• OBT vs IBT• Thinking and Testing Differently with IBT• Developing IBT Tests and App Maps• SAFS Image-Based Recognition Syntax• How to automate using IBT• Image Manager Tool• Sample Application Map And Script
Agenda:• Dealing with Variations in the IBT Environment• Dealing with NLS Testing in an IBT Environment• Challenges• Demo• Q & A
What exactly IS Image-Based Testing?
• Strictly “What You See is What You Get” (WYSIWYG)• Interaction with the visual display of the screen• Testing based on finding and interacting with graphics
on the screen• Automation is driven through keyboard and pointer
events• Test technologies partially or not supported by existing
toolso E.g Legacy Applications (Xterm application),Flash
Applications, GUI on Unix
What exactly IS Image-Based Testing?
• An alternative when items are not recognized at all by existing tools
• Independent of any underlying application technology
• The antonym or opposite of Object-Based Testing.
What exactly IS Image-Based Testing?
Image Based Testing Tool
Operating System
Application Under Test
DisplayBuffer
Mouse pointer events/ keyboard events queue
OBT vs IBT
Object-Based Testing• Ideal for automating PC
hosts • Performance: slower than
IBT• Can automate supported
technologies only• Can produce reliable &
robust scripts(time tested)• Wide range of functions
available to use
Image-Based Testing• Ideal for Applications which
OBT can’t support or limited support e.g. GUI on non PC hosts, legacy applications
• Performance :faster than OBT
• Can automate virtually anything that can be displayed on the screen
• Needs to be proved\improved after extensive usage
• Limited verifications
Thinking and Testing Differently for IBT
• Object-Based Testing deals with real components.• Image-Based Testing deals with images.• IBT doesn’t look for any object hierarchy • Application Map or Object Map is defined by images
not by properties• All possible Image variations needs to be stored
SAFS Image-Based Recognition Syntax
• SAFS keeps concept of Components in Windows
• A “component” can be: an image inside another image.
an image inside the bounds of other images.
Component as an Image inside another Image
Finding the right Arrow in a Sea of Matching Arrows• Locate the Unique Anchor that contains the Target• Locate the Target within the Anchor
Component Function call:BulletsItem, ArrowButton, Click
• A “component” can be: an image inside another image.
an image inside the bounds of other images.
BulletsItem ArrowButton
Component as an Image inside the bounds defined by other Images
Finding a Target inside the bounds from a single image Component Function call:OfficeWin, NewSlide, Click
• A “component” can be: an image inside another image.
an image inside the bounds of other images.
OfficeWin NewSlide
Component as an Image inside the bounds defined by other Images
Find a Target inside the bounds defined by multiple imagesComponent Function call:OfficeWin, CenterText, Click
• A “component” can be: an image inside another image.
an image inside the bounds of other images.
OfficeWin CenterText
SAFS Image-Based Recognition Syntax
• Component is an image inside another image. [BulletsItem]
BulletsItem =“Image=pathTo\BulletsItemImages” ArrowButton=“Image=pathTo\ArrowButtonImages”
• Component is inside bounds defined by other images. [OfficeWin]
OfficeWin =“Image=pathTo\OfficeAnchorImages; _ImageRight=pathTo\OfficeCloseIcons”
CenterText=“Image=pathTo\CenterTextImages”
BulletsItem
ArrowButton
OfficeWin
CenterText
How to automate using IBT• Test Development for IBT is the same
Fewer Component Functions supported. Tests and commands look the same as OBT.
• App Map development is mostly the same Recognition strings are VERY different.
It’s all about images, not components.e.g.IExplorer App Map Entry:[IExplorer]IExplorer="Image=<imagepath>"
How to automate using IBT
• Capture images in supported formats o BMP, GIF,JPEG,PNG,TIFF etc.
• Map logical names for the "window" and the "component" in the application map
• Write test records\scripts same as OBT• Execute records using SAFSDRIVER• Review the results
Image Manager Tool
• Tool to facilitate image captureso Command to launch
java org.safs.image.ImageManager
Enhancements
BitTolerance|BT=Optional. Specifies the integer percentage (1-100) of image bits or pixels that must match for an image to be considered a successful match. The default is, of course, 100. This means ALL pixels must match unless some other BitTolerance is specified.Samples:IExplorer="Image=<imagepath>;BitTolerance=70"IExplorer="Image=<imagepath>;ImageR=<imagepath>;BT=75"
Sample Application Map
[SampleApplication]SampleApplication="Image=c:\Images\AnchorImage.bmp;ImageR=c:\Images\CloseIcon.bmp"ButtonMinimize="Image=c:\Images\MinIcon.bmp"ButtonMaximize="Image=c:\Images\MaxIcon.bmp"ButtonClose="Image=c:\Images\CloseIcon.bmp"
Sample Test Records
C SetApplicationMap Demo.MAPC LaunchApplication SampleApplication "c:\safs\samples\Dotnet\DotNetApp\WinDemo.exe" C WaitForGUI SampleApplication SampleApplication 15 T SampleApplication SampleApplication GetGUIImage c:\OutputImage1.jpg T SampleApplication SampleApplication RightClick T SampleApp SampleApp InputKeys "x" T SampleApplication SampleApplication GetGUIImage c:\OutputImage2.jpg T SampleApplication ButtonClose Click
Dealing with Variations in the IBT Environment
• 32-Bit, 24-Bit, 16-Bit Color Depth and Resolution• Themes and Schemes• Platform and Version Image and Icon Variations • Multiple images are necessary if the target image is
different in different environments• Identify and Store Item Variations in a Directory
Dealing with NLS Testing in an IBT Environment
• Same as any Recognition String handling for NLS:Item1=pathTo\{^SHARED_DIR}\targetImages
Item2=pathTo\{^LANG_DIR}\targetImages
• Images will have to be captured and transferred if there is a language dependency
Challenges
• It’s all about imageso Require to capture all possible image variationso Precise\Reliable\Unique Imageso Display configurationso Time consuming test developmento Verification
Image verification only Can’t verify component properties Can’t extract data from application under test
• Themes and Schemes• Platform and Version Image and Icon Variations • Identify and Store Item Variations in a Directory
Demo
• What the IBT demo doeso Launch Sample Applicationo Perform few interactionso Closeo Review the results
Q&A
Thanks
• References :o SAFS Image-Based Recognition
http://safsdev.sourceforge.net/sqabasic2000/SAFSImageBasedRecognition.htm