model drivers how to industrialize testing 2013-09-01b
DESCRIPTION
Automation and repeat-ability - these are answers to the high cost of testing and business process conformance. These slides support a talk on how to structure and tool for testing and conformance industrialization.TRANSCRIPT
How to industrialize software testing
with Model Driven Testing and ModelDT tooling
How to industrialize testing2
About Model Drivers
Business model industrialization.
Model driven products and services: •Reporting•Testing•Regulatory conformance•Business strategy and business cases•Business architecture•Systems delivery
2013-09
How to industrialize testing3
Agenda
A.The testing landscape and its gapsB.Steps to industrialized testingC.How to tool up for industrialized testing with ModelDTD.Summary and further reading
2013-09
How to industrialize testing4
A: The testing landscape and its gaps
The development process is putting testing up the frontThere are many tools supporting the testing domainBut there is a problem – the tools don’t talk to each others
2013-09
How to industrialize testing5
The development process is putting testing up the front
• In the agile, Behaviour Driven Development, continuous delivery paradigms, test design comes before development.
• … therefore.. testing is even more closely integrate with business requirements
2013-09
User story Test case
Develop
Production
Backlog
User
Test result
The testing profile and its opportunity
6
There are many tools supporting the testing domain
• Testing is supported by many tools
• It has to be because testing's scope is 100% wide
• Many of the tools are excellent, with long, successful histories and wide adoption
2013-09
Quality Center
Auto testing of business rules
Behaviour driven testing
Testing environmentvirtualisation
Requirements management, test management
MD Testing specifications
Continuous testing and integration
MS Excel everywhere!
How to industrialize testing7
But there is a problem
2013-09
Project manager
User
Test environment
Test data
Unit test
Operating model /Business process
Risk management
Test plan
Test result
User AcceptanceTest
User story Test runner
DatabaseTester Programmer
Test caseSystem Under
Test
The tools do not talk to each other
How to industrialize testing8
B: Steps to industrialised testing
1. Adopt the vision and mental model2. Implement four big ideas3. Continuous learning and improvement
2013-09
How to industrialize testing9
1. Adopting the vision
What good testing looks like
Tests are 100% aligned to the requirements(even when things change)
Guaranteed correct
Fast (=automated)
Cheap (=automated)
Fully visible – at the tester level
Well governed and managed
2013-09
Industrialization: Conversion to the methods, aims, and ideals of industry and economic activity, particularly of an area that was previously underdeveloped
How to industrialize testing10
The model driven testing mental model
Architect
Design
Build
2013-09
1: Separation of concerns, with each concern well managed.
2: Testing analysis and design is supported with specific tools. language and designs.
How to industrialize testing11
2. Implement four big ideas
• Big idea 1: Industrialise your understand of your business• Big idea 2: Industrialise the design of your testing ecosystem• Big idea 3: Overlay your business understanding with your
testing requirements• Big idea 4: Drive testing through automation and tooling
2013-09
How to industrialize testing12
Step 1: Industrialize your understanding of your business
1. You cant do a good job of testing without a good understanding of the requirements
2. To industrialise testing you therefor have to industrialise your understanding of the business domain
3. This means tooling and appropriate language
2013-09
Business process model Data model
How to industrialize testing13
Step 2: Industrialise the design of your testing ecosystem
1. Industrialized testing can only be built from a structured, automated design
2. That requires a specific, specialised language
3. And supporting tooling
4. Within a well specified framework
2013-09
Testing eco-system
..break your analysis and design of testing into manageable packages
2013-09 How to industrialize testing14
Test architecture: Testing meta model, business domain, testing packageTest package: System Under Test, test context, test componentsTest configuration: Test cases, scheduler, arbiters, logsTest case: Testing activity
Test data: Data pools, partitions, selectors Deployment: Nodes for SUT, Test components, Scheduler
The UML Testing
Profile gives the testing industry 6
domains to standardize
on
How to industrialize testing15
Step 3: Overlay your business understanding with your testing requirements
2013-09
Business domain Testing domain
Pro
cess
Dat
a
Data model
Process model
Generation
Generation
Overlay the testing
strategy
Test cases
Test data
Test strategy
How to industrialize testing16
Step 4: Drive testing through automation and tooling
2013-09
Because design information is structured, this interface
can be automated
Quality Center
Auto testing of business rules
Behaviour driven testing
Testing environmentvirtualisation
Requirements management, test management
MD Word testing specifications
Continuous testing and integration
MS Excel everywhere!
Business and systems modelling
How to industrialize testing17
C: How to tool up for industrialized testing with ModelDT
Tooling for planning, analysis and designExample automations with ModelDT
2013-09
How to industrialize testing18
Tooling for planning, analysis and design
2013-09
Manager & Architect
Project manager
User
Test environment
Test data
Unit test
Operating model /Business process
Risk management Test plan
Test result
User AcceptanceTest
User story
Test runner
DatabaseTester Programmer
Test caseSUT
You get Architectural view of
testing Large scale automation
How to industrialize testing19
Example automations with ModelDT
a) Harvest and integrate existing knowledgeb) Validation to the testing profile rulesc) Analysis toolsd) Auto generate test plans and specificationse) Auto generate code that tests your data to the requirementsf) Auto generate test cases and scripts for test runnersg) Synthetic test data generationh) Work in familiar Web and Excel for day to day work
2013-09
How to industrialize testing20
a) Harvest and integrate existing knowledge
2013-09
Quality Center
You get Synthesis De-duplication Consistency Visibility
How to industrialize testing21
b) Validation to the testing profile rules
2013-09
You get Highlights on items in error Error messages describing the break Guaranteed conformance to the UTP rules
How to industrialize testing22
c) Analysis tools
2013-09
Business domain – use case and data entities
Test
com
pone
nts:
E.g
. te
st c
ase
Coverage of business domain By testing package
You get Management reports on test coverage Control reports Change impact assessments
How to industrialize testing23
d) Auto generate test plans and specifications
2013-09
Test package design
Automated test specifications
You get Consistency and quality Guaranteed alignment between
business domain and tests Automated specs and designs
Deployment plans
The testing profile and its opportunity
24
e) Auto generate code that tests your data to the requirements
You get Automated testing of data format and
business rules Guaranteed alignment between
requirements and tests
Data format validations
Constraints
Class diagram
2013-09
Business rule validations
Schema
How to industrialize testing25
f) Auto generate test cases and scripts for test runners
2013-09
Junit
Quality Center
Business data models
Messages and signals
Business processes
Process processes
Test casesTest runners
Test scripts
How to industrialize testing26
g) Synthetic test data generation
2013-09
Upload data and business
rule definitionsDownload
comprehensive test data
Business domain models
Positive test data
Negative test data
ModelDT web service
You get Data with 100% test coverage Guaranteed alignment of
business and test data Test data fast
How to industrialize testing27
h) Work in familiar Web and Excel for day to day work
2013-09
Web interface
Excel interface
Modelling interface
Database
RE
ST
ful
web
se
rvic
e in
terf
ace
The testing profile and its opportunity
28
modelDT plugin development roadmap
1.0–Base Domain Specific Language–Test Case designs –Business domain coverage reports–Synthetic test data generation–End user help, manual, training
1.1–Extended DSL–Extended analysis reports
2.0–Test script outputs – Junit, Quality Centre, etc–Methodology wizards–Domain specific patterns and models
3.0–Risk modelling and management
2013-09
How to industrialize testing29
D: Summary and further reading
Summary- Testing is supported by many great tools- But they do not talk to each other- The UML Testing profile is a testing language- ModelDT is a tool that implements the UML Testing Profile- Thus enabling integration and large scale automation
2013-09
How to industrialize testing30
Next steps
• The UML Testing Profile: Slide presentation• Demonstration: Attend a short web presentation and
explore any aspect of Model Driven Testing.• White paper: Request a copy of “Roadmap to
industrialised testing”. • Questions: Greg Soulsby http://uk.linkedin.com/in/gregsoulsby
• Twitter: @modelDT, @modelDrivers• Youtube: modelDrivenTesting• Linked in group: www.linkedin.com/groups/Model-drivers-4341672/
• Blog: http://blog.modeldrivers.us/• Site: www.modelDrivers.us• Plugin beta: Get the benefits with free support &
contribute to direction
2013-09