verifying web applications: from business level specifications to automated model-based testing...
TRANSCRIPT
![Page 1: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/1.jpg)
Verifying Web Applications: From Business Level Specifications to
Automated Model-Based Testing
Christian Colombo, Mark Micallef, Mark Scerri
![Page 2: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/2.jpg)
Health System
![Page 3: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/3.jpg)
Verifying the Health System
• Important to check implementation(not only the model)
![Page 4: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/4.jpg)
Verifying the Health System
• Important to check implementation(not only the model)
• Testing and runtime verification
![Page 5: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/5.jpg)
Manual Testing
![Page 6: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/6.jpg)
Manual Testing
Laborious and slow!
![Page 7: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/7.jpg)
Manual Testing
Laborious and slow!
Automate?
![Page 8: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/8.jpg)
Automated Testing Tools
System
Executable Specification
![Page 9: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/9.jpg)
Automated Testing Tools
SystemTest Cases
Executable Specification
![Page 10: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/10.jpg)
Automated Testing Tools
SystemTest Cases Oracle
Executable Specification
![Page 11: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/11.jpg)
QuickCheck – Automated Random Testing
pre
post
action
pre postaction
etc
![Page 12: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/12.jpg)
QC for Health System
![Page 13: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/13.jpg)
Sample Test Case
![Page 14: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/14.jpg)
Connecting QC to Health System
SystemTest Casesgenerated
by QC
Post condition
check
Executable SpecificationFSM
Click
Read text area
Bridge
![Page 15: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/15.jpg)
Connecting QC to Health System
SystemTest Casesgenerated
by QC
Post condition
check
Executable SpecificationFSM
Bridge
But where does the executable specification come from?
![Page 16: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/16.jpg)
Connecting QC to Health System
SystemTest Casesgenerated
by QC
Post condition
check
Executable SpecificationFSM
Bridge
But where does the executable specification come from?
How complete is the specification?
![Page 17: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/17.jpg)
Standard Business Specifications
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
![Page 18: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/18.jpg)
Standard Business Specifications
![Page 19: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/19.jpg)
Standard Business Specifications
States
![Page 20: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/20.jpg)
Standard Business Specifications
States
Actions
![Page 21: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/21.jpg)
Standard Business Specifications
Post Condition
Pre Condition
![Page 22: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/22.jpg)
Automatically Generated QC Model
![Page 23: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/23.jpg)
Whole Process
Cucumber QuickCheck Model Bridge System
![Page 24: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/24.jpg)
Results: Functionality Covered
• Out of 43 function points, 39 were covered• Limitations of our language/architecture• Eg: checking a PDF, other intricate aspects
CoveredNot Covered
![Page 25: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/25.jpg)
Results: Test Cases
• 204 manual test cases• 150 out of 204 created automatically– Remaining 54 could not be created due to non-
covered functionality
CoveredNot Covered
![Page 26: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/26.jpg)
Results: Time Taken (days)
Manual
Automatic
0 1 2 3 4 5 6 7 8
Bridge Setup (manual)
Cucumber Specifications (manual)
Test Case Generation (manual)
Test Case Generation (automatic)
Test Case Execution (manual)
Test Case Execution (automatic)
![Page 27: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/27.jpg)
Results: Time Taken (days)
Manual
Automatic
0 1 2 3 4 5 6 7 8
Bridge Setup (manual)
Cucumber Specifications (manual)
Test Case Generation (manual)
Test Case Generation (automatic)
Test Case Execution (manual)
Test Case Execution (automatic)
manual
![Page 28: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/28.jpg)
Conclusions
• Completely automated testing • No pain to come up with specification
![Page 29: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/29.jpg)
Future Work
• Testing will not cover all possibilities• Use test checks during runtime
![Page 30: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/30.jpg)
Runtime Verification
• Testing will not cover all possibilities• Use test checks during runtime• QuickCheck specifications can also be used for
runtime verification
![Page 31: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/31.jpg)
QuickCheck – Action Generation
pre
post
action
pre postaction
etc
![Page 32: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/32.jpg)
QuickCheck – Event Listeners
pre
post
event
pre postevent
etc
![Page 33: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/33.jpg)
OptimisationWell tested...
Do not runtime verify
Keep verifying at runtime
![Page 34: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/34.jpg)
Future Work
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
![Page 35: Verifying Web Applications: From Business Level Specifications to Automated Model-Based Testing Christian Colombo, Mark Micallef, Mark Scerri](https://reader036.vdocuments.site/reader036/viewer/2022070306/55163512550346b2068b4e55/html5/thumbnails/35.jpg)
Giving Back to Business Specification World
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
Given: ...When: ...Then: ...And: ...
Extract common checks