![Page 2: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/2.jpg)
Testing as a processg prequirements Planning&
Control
olog
i
specificationpreparation test plan
tekn
o
(abstract)Test
generation
abstract/logicaltest casesspecification
tions
t generation
Testimplemen-
tation
concrete/executableltest cases
orm
at
implementationTest
execution verdict/logs
Info
quality reportCompletion
CISS
![Page 3: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/3.jpg)
Test Automationrequirements Planning&
Control
olog
i
specificationpreparation test plan
tekn
o
(abstract)Test
generation
abstract/logicaltest casesspecification
tions
t generation
Testimplemen-
tation
concrete/executableltest cases
orm
at
implementationTest
execution verdict/logs
Info
quality reportCompletion
CISS
![Page 4: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/4.jpg)
1 Manual Testing1. Manual Testing+ easy & cheap to start+ flexible testing- expensive every execution
i i- no auto regression testing- ad-hoc coverage- no coverage measurement- no coverage measurement
M. Utting ”Practical Model Based Testing”
![Page 5: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/5.jpg)
2 Capture Replay Testing2. Capture-Replay Testing+ flexible testing+ flexible testingexpensive first execution+ auto regression testing- fragile tests break easily- ad-hoc coverage- no coverage measurement
![Page 6: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/6.jpg)
3 Script Based Testing3. Script-Based Testing+/- test impl. = programming+ automatic execution+ auto regression testing+ auto regression testingfragile tests break easily?
(depends on abstraction)(depends on abstraction)- ad-hoc coverage- no coverage measurement
![Page 7: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/7.jpg)
4 Keyword Driven Testing4. Keyword-Driven Testing+ abstract tests+ automatic execution+ auto regression testing+ auto regression testing- robust tests- ad-hoc coveragead hoc coverage- no coverage measurement
![Page 8: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/8.jpg)
5 Model-Based5. Model BasedTestingg
+ abstract tests+ automatic execution+ auto regression testing+ auto regression testing+ auto design of tests+ systematic coverage+ systematic coverage+ measure coverage of model and requirements- modelling overhead
![Page 9: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/9.jpg)
Script based TestingScript-based Testing
Test inputs and observations (verdictassignment) programmed in scripts in g ) p g pdedicated or general-purpose languages.
+ Repeatable (for regression testing)+ Repeatable (for regression testing)+ Data driven testing: Fixed scripts, each parameterized
with different datad e e da a– Mixes test harness and logical test cases– Must be updated when specs (or impl) changeMust be updated when specs (or impl) change– Very lengthy– Good programmers may write well-structuredGood programmers may write well structured
reusable test code but ”it is just test code”
![Page 10: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/10.jpg)
Keyword Driven Testingy g• Script based automation where test case
d i i t d f t tidesign is separated from automation– Focus on actions user/environment can do on
objects in SUT (at different abstraction levels)• ”Actions” appear in scripts,pp p• ”Action-code” implements the action
• aka “Action-word” testing (or table driven• aka Action-word testing (or, table driven testing), < 1994
• Black-box, subsystem, accept tests• Automated execution using a “framework”.Automated execution using a framework .
![Page 11: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/11.jpg)
Keyword Driven Testingy g+ Concise, flexible, maintainable, + Read-/writeable by non-programmers
÷ Expressiveness of a scripting languageC t l t t d l t ti– Control structures and complex computations
– (Branching in test cases, complex data, matching of expected results)
![Page 12: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/12.jpg)
Keyword Driven TestyObject Keyword Value
Eg. Sequence
LoginDialog Login ”badUserId”, ”GoodPass”
VerifyLogin ”Login failed”VerifyLogin Login failed
LoginDialog Login ”goodUserId”, ”badPass”
VerifyLogin ”Login failed”
LoginDialog Login ”goodUserId”, ”GoodPass”
VerifyLogin ”welcome”
W l Di l h L i ” U N ” ” d”WelcomeDialog changeLogin ”newUserName”,”newpwd”
confirm ”goodUserId”, ”newpwd”
confirmChange ”Change Failed”
![Page 13: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/13.jpg)
KWD ProcessTest Expert Automation Expert
http://www.softwaretestinghelp.com/wp-content/qa/uploads/2010/01/keyword-driven-testing.pdf
![Page 14: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/14.jpg)
Automation FixtureAutomation FixtureKeywod ScriptKeywod Script
Open(Scripts)
Engine/driverGeneric Libs
(fixtures)
While(!EOF){ Keyword, data=readFile();Case(keyword){L i id t > b it(d t )(fixtures) Login: widget->submit(data);
verifyLogin: …
Application Libs(keyword functions)
}Close()
SUT
}
![Page 15: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/15.jpg)
Tools• SAFS, open source• EMOS Framework, open source• FIT/FitNesse open source• FIT/FitNesse, open source• Certify, Worksoft• Unified TestPro, SDT• TestFrame LogicaCMG• TestFrame, LogicaCMG• TestArchitect, LogiGear• Tools with built-in keyword support:
TestQuest Pro– TestQuest Pro,– QuickTest Professional
![Page 16: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/16.jpg)
ExampleExamplehttp://localhost:8080/FitBookExamples.Chapter4TestingActions.Fig6TestChatServer
![Page 17: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/17.jpg)
http://fit.c2.com/wiki.cgi?FitBookhttp://my.safaribooksonline.com/0321269349
![Page 18: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/18.jpg)
ComparisonpCriteria Manual Script KWD Model-Based
Cost Development L M M LCost Development L M M L
Maintenance H? H L
Tools/equip L M M Hq p
Time Time to first test L M M M-H
Re-execution H L L L
Maintenance L H H L
Learn-ability
Skills L H M* H*ability Maturity L M M-H H
Readability L L M H*
I t t / Ch ll i L M M HInterest / Challenging L M M H
Effective-ness
Bug detection H L L H
Tracability L L M HTracability L L M H
Coverage L L M H
![Page 19: Test Automation and Keyword-driven testing af Brian Nielsen, CISS/AAU](https://reader035.vdocuments.site/reader035/viewer/2022062405/5568d3b2d8b42a173c8b4f8b/html5/thumbnails/19.jpg)
ReferencesReferences• http://en.wikipedia.org/wiki/Keyword-driven_testing• http://www.cs.waikato.ac.nz/~marku/mbt/• http://www.onestoptesting.com/automation-
framework/keyword-driven-testing/• http://www.softwaretestinghelp.com/wp-
content/qa/uploads/2010/01/keyword-driven-testing.pdf• http://www.musala.com/press/PR_2008_03_17/pr/Keyw
d d i %20T ti dford-driven%20Testing.pdf• Mark Fewster and Dorothy Graham, Software Test
Automation 1999 (Chapter 22)Automation, 1999 (Chapter 22) • M. Utting Practical Model-based testing