automating jfc ui application testing with jemmy
DESCRIPTION
В докладе рассказано о нескольких подходах к автоматизации тестирования через пользовательский интерфейс. Вы узнаете как автоматизировать приложения на Java Swing. Также будет рассмотрен инструмент автоматизированного тестирования Jemmy, продемонстрирована работа с ним. Еще вы познакомитесь с новыми возможностями Jemmy 3.TRANSCRIPT
![Page 1: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/1.jpg)
<Insert Picture Here>
Automating JFC UI application testing with Jemmy.
Alexandre (Shura) IlineJava SE and JavaFX Quality architect.
![Page 2: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/2.jpg)
The following is intended to outline our general product direction. It is intended for information purposes, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle.
![Page 3: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/3.jpg)
This presentation shares experience gotten from testing such products as
JRockit JavaFX AT
![Page 4: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/4.jpg)
What's in scope
• All sorts of definitions “Testing”, “UI testing”, “Test automation”• Jemmy library• Automation approaches effectiveness and cost.• Bunch of source code
• Basic Jemmy operations• Less basic operations• Switches and settings
• Tricks and tips• Misconceptions
![Page 5: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/5.jpg)
What's not in scope
• The economical effects• A.k.a. what's “quality”• Why to automate
• Automation cost vs. manual testing cost• When to automate• What to automate
• Non-economical benefits of automation
A.k.a. let's leave it for later. :)
![Page 6: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/6.jpg)
UI testing … by Wikipedia
«GUI software testing is the process of testing a product that uses a graphical user interface, to ensure it meets its written specifications.»
![Page 7: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/7.jpg)
UI testing … most often ...
«Checking whether usage of a product UI leads to results expected by the the person who performs testing»
![Page 8: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/8.jpg)
●Start text editor●Push «File/Open»●Verify file chooser directory●Select some file●Verify editor area content●Verify application title●Verify buttons availabilities●....
Sample test scenario
![Page 9: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/9.jpg)
Find Do VerifyPass Pass
Pass
Fail Fail Fail
Failure analysis
Find next controlTo perform operation
Verify that expectedState reached
Perform necessary actions
UI test workflow
![Page 10: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/10.jpg)
Test automation ... for me
«Building a process which exercises and reports certain product characteristics while run unattended.»
![Page 11: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/11.jpg)
Test automation is like development
• Putting logic in a code• Same lifecycle:
• Requirements• Design• Implementation
• Same set of problems• Bugs • Instabilities
• Scope • defined through test specification/plan
![Page 12: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/12.jpg)
Test automation is not like development
• Big fat dependency – the tested product• vs many libraries and platform
• Many small programs• vs one big program
• Does one thing good – reports status• vs does many thing ... good
• Perfectness is not the goal• other than maintenance cost, ease of use
![Page 13: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/13.jpg)
Jemmy
![Page 14: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/14.jpg)
Jemmy history• Started as a tool to tests TeamWare UI (1999)• Used for NetBeans extensions (2000)• Official test tool for NetBeans (2001)• Open-source (2001)• Is used for NetBeans and extensions since then• Adopted as a test tool for Swing (~2003)• Used outside SUN Microsystems (next slide)• Jemmy v3 created (2008)• The test tool for JavaFX SDK (2008 – now)• Extended to support SWT• Test tool for JRMC
![Page 15: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/15.jpg)
JemmyCore
JemmySGJemmy v2
AWT & Swing
JemmyRemote
JemmyFXRemoteJemmyFXJemmySWT
JRMCJavaFX Authoring Tool
JavaFX
LCDUINetBeans
JDK
Jemmy and extensions
Jemmy v3
JemmyAWT
![Page 16: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/16.jpg)
Jemmy v2 JFC
![Page 17: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/17.jpg)
Same VM.
Test code runs in the same VM as the application code• Benefits
• Full access to the application UI objects• As well as the application domain object (sometimes).• Control over event queue
• Drawbacks• Impacts the tested UI
• Options• Run application from test• Run test from application• Use accessibility hook
![Page 18: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/18.jpg)
Using Jemmy
Demo
![Page 19: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/19.jpg)
Operators
Test codeTest code
JFrameOperatorgetTitle(),
...
JFrameOperatorgetTitle(),
...
JComboBoxOperatorgetSelectedItem(),
selectItem(...)
JComboBoxOperatorgetSelectedItem(),
selectItem(...)
JTextFieldOperatorgetText(),
typeText(...), ...
JTextFieldOperatorgetText(),
typeText(...), ...
JButtonOperatorgetText(),
push()
JButtonOperatorgetText(),
push()
![Page 20: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/20.jpg)
Operators
Demo
![Page 21: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/21.jpg)
Threading
“noblocking”operation
periodicalchecks
waiting waitingaction action action
action
![Page 22: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/22.jpg)
WaitingTimeouts
Demo
![Page 23: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/23.jpg)
Event queue
User
UIOtherthreads
![Page 24: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/24.jpg)
UIOtherthreads
Test
Actions
Verifications
Lookup
Event queuewith test
![Page 25: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/25.jpg)
Event queue tools
Demo
![Page 26: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/26.jpg)
Drivers
Test code
ButtonDriverpush(...)
...
TextDrivertypeText()
...WindowDriver
ListDriverslectItem(...)
...
JButtonOperator
JTextFieldOperator
JComboBoxOperator
JFrameOperator
![Page 27: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/27.jpg)
Drivers
Demo
![Page 28: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/28.jpg)
Dispatching modes
• Events• Proper events in a proper order are dispatched to a
component making it thinking some user actions are performed.
• Shortcut• Same as events except the events are posted into the event
queue at once. • Robot
• java.awt.Robot is used
Implemented with driver sets
![Page 29: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/29.jpg)
Dispatching models
Demo
![Page 30: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/30.jpg)
Image
• Fragile• Needed for custom controls
• Comparison• Pixel-to-pixel• %s of pixels• Color distance
• Average• Maximum
![Page 31: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/31.jpg)
Verifying images
Demo
![Page 32: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/32.jpg)
Automationapproaches
![Page 33: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/33.jpg)
33
33
Application UI
Product UIProduct UI
![Page 34: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/34.jpg)
34
34
Coordinates
• click(134,32) //selects some record• click(215,122) //hits “Properties”• sleep(5) //sleeps to let dialog be painted• click(64,182) //expands color combo• click(235,182) //selects Gray• click(235,212) //hit OK
![Page 35: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/35.jpg)
35
35
Widgets
• Find “Car records” frame• Find table• Select “1abc234” cell• Push “Properties” button• Wait for “1abc234” dialog• Select “Gray” color in combo box• Push “OK”
![Page 36: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/36.jpg)
36
36
Prod
uct U
IPr
oduc
t UI
Dom
ain
mod
el
Dom
ain
mod
el
Car record
ColorColorModelModel MakeMake YearYear License plateLicense plateVINVIN
Test
Widgets or coordinates
![Page 37: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/37.jpg)
Cars testWidgets
Demo
![Page 38: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/38.jpg)
38
38
UI Primitives
• Find car list frameCarListFrame list = new CarListFrame()
• Open properties dialog for car “1abc234”CarDialog propDialog = list.carProperties(“1abc234”);
• Set color to graypropDialog.setColor(Color.GRAY);
• Apply changespropDialog.ok();
![Page 39: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/39.jpg)
39
39
LibraryPr
oduc
t UI
Prod
uct U
ID
omai
nm
odel
Dom
ain
mod
el
Car record
Test libraryTest library
ColorColorModelModel MakeMake YearYear License plateLicense plateVINVIN
CarListFrame CarDialog
Test
![Page 40: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/40.jpg)
Cars testUI Library
Demo
![Page 41: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/41.jpg)
41
41
Domain model
• Set color to gray for a car “1abc234”
new CarRecord(“1abc234”). setColor(Color.GRAY);
Underneath the cover, CarRecord class does all described earlier
![Page 42: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/42.jpg)
42
42
Domain libraryPr
oduc
tU
IProd
uct
UI
Dom
ain
mod
el
Dom
ain
mod
el Car record
Domain test libraryDomain test library
ColorColorModelModel MakeMake YearYear License plateLicense plateVINVIN
UI test libraryUI test libraryCarList CarDialog
CarRecord
Test
![Page 43: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/43.jpg)
Cars testDomain Library
Demo
![Page 44: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/44.jpg)
Automation effectiveness
![Page 45: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/45.jpg)
The formula?
TD + *T
S NR
TM * N
RN
C*E
A =N
C*
EA – automation effectiveness
To be used for every particular product.
NR and N
C are unique for a product.
TM is a characteristic of a test suite.
Smaller TD and T
S - higher the E
A.
Coefficient depend on the way you write your tests
![Page 46: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/46.jpg)
Coordinates Widgets UI Library Domain library0
1
2
3
4
5
6
7
8
1.1
3
5
7.5
10.5
0.1 0.05
Td/TmTs/Tm
Td and T
s together
![Page 47: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/47.jpg)
Coordinates Widgets UI Library Domain library0
5
10
15
20
25
30
1.13
5
7.5
24
12
2.41.2
25.1
15
7.48.7
TdTsTd+(Ts*Nc*Nr)
TD and T
S for N
C=3, N
R=8, T
M=1
![Page 48: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/48.jpg)
Coordinates Widgets UI Library Domain library0
0.5
1
1.5
2
2.5
3
3.5
0.96
1.6
3.24
2.76
Ea
EA for N
C=3, N
R=8
![Page 49: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/49.jpg)
Misconceptions
• Automated testing replaces manual testing• it will find all the bugs• it will find more bugs• it does the same amount of work
• Create once use forever• This is easy• This is too hard• No need to create test plan• “Will just buy the XXX tool and it will solve all out
problems”
![Page 50: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/50.jpg)
Resources
jemmy.dev.java.net
![Page 51: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/51.jpg)
<Insert Picture Here>
Automating JFC UI application testing.With Jemmy.The technical aspects. Alexandre (Shura) IlineJava SE and JavaFX Quality architect.
![Page 52: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/52.jpg)
Backup slides
![Page 53: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/53.jpg)
Coverage
• Implementation coverage• Line, block, condition, sequence, ...
• Specification coverage• How well the tests covering the functional specification
• Public API coverage• Whether the public API is tested fully
• UI coverage• Whether the tests cover UI fully
• Combine with bug density• First cover the areas with more bugs
• Combine with code complexity• First cover the more complicated code.
![Page 54: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/54.jpg)
Development
Continuous build
Commit
BuildExecuted
automaticallyafter commit
Success
Analysis.Rollback!!!
Code changes
NoYes.
PromoteCode is compilableTest further
![Page 55: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/55.jpg)
Continuous build with testing
Commit
Build Success
TestingIs it working?
Passed
Analysis.
Rollback!!!
Code changes
Testchanges
No
No
Test furtherBuild is good
Code line is healthyGo on ...
Yes
Yes. = Compilation
successful
![Page 56: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/56.jpg)
Pre-integration
Commit
Code changes
Testchanges
Testing Passed
No
Yes
![Page 57: Automating JFC UI application testing with Jemmy](https://reader035.vdocuments.site/reader035/viewer/2022081801/556bfa53d8b42a6d768b48a9/html5/thumbnails/57.jpg)
How to benefit from UI automation?Use it!
• Sanity• Pre-integration• Attach to bug reports• Make is a quality criteria• Run it for every build• Show it to the boss :)• Don't forget to show it to the dev. team