testing gripen e – test automation and exploratory testingtddc88/theory/presentation liu oct...
TRANSCRIPT
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
This document and the information contained herein is the property of Saab AB andmust not be used, disclosed or altered without Saab AB prior written consent.
Testing Gripen E –Test Automation andExploratory Testing
Torvald MårtenssonTest Manager Gripen E/F M1, Technical Fellow
Linköping University, October 2 2020
1
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Who am I?
• Torvald Mårtensson PhD [email protected]
• Test Manager in the Gripen E systemsdevelopment project at Saab
2
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Takeaways from this talk
• What is Gripen E?
• Testing a large-scale and complex softwaresystem – how is it done?
• Three examples of practices implemented inthe Gripen E project
3
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Gripen A/B and C/D
• Gripen A (single seater) and B (dual seater)was developed for the Swedish Air Force
• Gripen C and Gripen D are currentlyoperational in SE, CZ, HU, TH and SA
4
PAGE 25Torvald Mårtensson
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Gripen E – a new aircraft
5
First flight for the first Gripen E test aircraft (June 2017)
First flight for the first Brazilian Gripen E aircraft (August 2019)
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Developing Gripen E
6
Training Systems
Support Systems
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Developing Gripen E
7
Training Systems
Support SystemsLarge-scale Complexity
Testing a large-scale and complex software system
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Testing on different levels
8
Testing asoftware
component
Testing asubsystem
Testing thecomplete
system
Manual testing(scripted and exploratory)
Automated testing
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Testing in different types of test environments
9Simulation of targets and threats
Simulation or real hardware?
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Testing in different types of test environments
10
Hardware in the loopsub-system rigs
Test aircraft
Software in the loopsimulators
Hardware in the loopsystem rigs
Example 1 Example 2
Real hardware
Simulation
Real hardwareSimulation
Real hardware
Simulation
Real hardware
Simulation
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Practices implemented in the Gripen E project
11
• Testing in the continuous integrationand delivery pipeline
• A simulator on every developer’s desk
• Exploratory testing of the completesystem
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
The continuous integration and delivery pipeline• Creating a pipeline of test activities which evaluate
software changes made by the developers• Example:
• Design the pipeline with…• Unit/component tests and system tests• Automated testing and manual testing• Testing in simulators and on real hardware
• There is no optimal pipeline for every product andevery organization
• Designing the pipeline with a systematic approachrather than making changes and hoping for the best
12
Duvall: “Stage builds”
Larman and Vodde: “multi-stage CI system”
Humble and Farley: “deploymentpipeline” or “integration pipeline”
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
A simulator on every developer’s desk
• The MySim simulator is available for alldevelopers from the developer’s own desk
• The developers are more confident that theirsoftware is working
• MySim is also used for automated regressiontesting on different levels of the product
• Faster feedback to the developers13
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Exploratory testing of the complete system
• Exploratory testing with experienced engineersrepresenting different subsystems
• The test teams identify complex integration problemswhich are difficult to find with automated testing
• The developers test the complete system together –not just a software module or a subsystem
14
“We could test according to the ideas we had. We wantedto understand the system that we were building and to findthe weaknesses in the system”.
“The quality of the problem reports improvesif there are people from different subsystemsparticipating at the test”.
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
Summary and takeaways
• What is Gripen E?• A completely new aircraft• A large-scale and complex software system
• Testing a large-scale and complex softwaresystem – how is it done?
• Testing on different levels• Testing in different types of test environments
• Three examples of practices implemented inthe Gripen E project• Testing in the continuous integration and delivery pipeline• A simulator on every developer’s desk• Exploratory testing of the complete system
15
COMPANY RESTRICTED | NOT EXPORT CONTROLLED | NOT CLASSIFIEDYour Name | Document Identification | Issue 1
16