![Page 1: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/1.jpg)
![Page 2: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/2.jpg)
Testing
• Testing is a critical process within any software development and maintenance life cycle model.
• Testing is becoming increasingly important to the software development industry because of its implications on the quality of the produced software and the reputation of the producing company.
• Testing activities include testing specifications, designs and implementations.
• Testing implementations is often referred to as software testing, or simply testing.
![Page 3: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/3.jpg)
Test coverage
• Only exhaustive testing can guarantee full coverage.
• Exhaustive testing is impractical since it may take years to complete even for testing small size programs.
• Consider testing the software when a first name text field of 20 characters maximum is filled by a user. Exhaustive testing requires that we test the software for all possible correct and incorrect field input values. – Assume 80 possible correct and incorrect characters, the number of
possible combinations for a valid or invalid first name would exceed 8020. Using a computer that takes, say, 50 nanoseconds to process one combination, we would need about 1011 years to complete the testing for all possible combinations. It is indeed impractical!
![Page 4: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/4.jpg)
The reality
• With the absence of exhaustive testing, there are no formal guarantees that software is error-free.
• In fact, it may happen in practice that even when all errors have been detected during testing, they may not all be fixed due to the lack of time.
• Unfixed errors may be fixed in future releases of the software.
• The difficulty in software testing is the inability to quantify the errors existing in software, and hence predict how many errors are left undiscovered.
![Page 5: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/5.jpg)
Test coverage experiment
• Random testing experiment: errors of different levels of complexity were inserted in the software under test
• All errors except for one were discovered progressively.
• No indication on how much more testing is needed
• Finding errors is an asymptotic nonlinear function of time.
![Page 6: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/6.jpg)
Dynamic and static testing
• Dynamic testing
– Test the software by executing it and applying test inputs
– Observe the output and make a judgment to take a verdict
• Static testing
– Check code without executing it
– Automatic static testing: Code as an input to a static analysis tool
– Non-automatic static testing: review, walkthrough, inspection
![Page 7: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/7.jpg)
Test case
• Test plan includes a test suite
• A test suite is a set of organized test cases
– Organize around use cases for example
• Template to describe a test case
– Id, name, reference, priority, preconditions, preamble, input, expected output, observed output, verdict and remarks
• Test cases are obtained using known testing techniques and strategies
![Page 8: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/8.jpg)
Test case design vs execution
• Test case is developed first – verdict and remarks are empty
• Test case is executed when the actual testing is performed – the verdict is obtained
• Typically test cases are developed (designed) before even writing any code
• Acceptance test cases in the acceptance test plan (ATP) can be developed after requirements specification phase
• Unit and integration test plans can be developed after the design is completed and prior to coding
![Page 9: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/9.jpg)
![Page 10: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/10.jpg)
![Page 11: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/11.jpg)
OOP Development 15
Test Case (TC) Example
TC#: S-221
TC Description: This test is to check the response to an invalid input selection.
TC Precondition: Go to screen Flight_Management
TC Input: Enter <F7>
TC Expected Output: Error message: “Invalid input, Enter “1” to “4” or ESC”
![Page 12: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/12.jpg)
White-box, black-box and grey-box testing techniques
• White-box: module or unit testing
• Black-box: module testing and system testing
![Page 13: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/13.jpg)
Black-box unit/module testing techniques
• Boundary value analysis
• Equivalence class testing
• Decision table / decision tree based testing
• State based testing
• Robustness testing
![Page 14: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/14.jpg)
Boundary value analysis
• BVA: test inputs obtained from the boundaries
– Serious errors hide at the boundaries.
• For example, simple boundary values for an input of integer data type would include -INT_MAX, -INT_MAX + 1, -1, 0, +1, INT_MAX - 1 and INT_MAX.
• For complex and composite data types, boundary values become more difficult to generate and may require more ingenuity to obtain them.
• Moreover, if interface specification includes more than one input, different test cases considering possible feasible combinations of BVs for each input must be considered
![Page 15: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/15.jpg)
BVA Example
• Suppose you are given an executable code including a function called digitCount whose functional interface specification states that digitCount takes an integer n as an input parameter, and returns an integer representing the number of digits in n.
• Test cases whose test inputs are selected based on boundary value analysis are partially shown in Table 8.3.
• Assume that for a 32-bit integer, INT_MAX is +2147483647.
![Page 16: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/16.jpg)
Testing function digitCount based on boundary value analysis
![Page 17: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/17.jpg)
Example 8.2 – Testing search function
![Page 18: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/18.jpg)
![Page 19: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/19.jpg)
Equivalence class testing
• Partitioning input domain into disjoint subsets or partitions
• Partition includes related values
• Consider one test input from each partition
• Equivalence relation among partition elements
– ‘having the same number of digits’
– ‘all positive integers’
– ‘all even numbers’
– ‘all prime numbers’
– ‘all powe or 2 numbers’
– …..
![Page 20: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/20.jpg)
![Page 21: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/21.jpg)
Decision table based testing
• Decision table: each column denoting a rule becomes the basis of a test case
– Conditions produce inputs
– Actions produce expected outputs
– In table 4.1: input variables are age, history and speed, and output variables are penalty and points
![Page 22: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/22.jpg)
![Page 23: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/23.jpg)
Decision tree-based testing
• Each distinct path from the root node to a leaf node is the basis of a test case
• If along the path, a variable input value does not exist, the test input for that variable can be assigned any valid value.
• For example, in fig 4.23, for the test case corresponding to the leftmost path, the speed value can be any illegal speed
![Page 24: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/24.jpg)
![Page 25: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/25.jpg)
State-based testing
• An implemented finite state machine conforms to its specification, if the observed output sequence matches the expected output according to the specification
• Test sequence must cover all states transitions
• Used to test do black box testing of systems
– GUI, protocols, web page connectivity, …
• Transition tour: a sequence covering all transitions and returning back to the initial state
– Can only cover output errors
![Page 26: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/26.jpg)
![Page 27: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/27.jpg)
Black box-based robustness testing
• Test a system using one or preferably more black box based testing method
• Test for incorrect and illegal inputs also
• In equivalence class testing, select one representative bad input from every partition of the bad input values
• For FSM based systems, apply unexpected inputs at each state – the system should survive
• For boundary value testing, test for illegal values beyond the boundaries – the system should survive
![Page 28: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/28.jpg)
White-box testing techniques
• Given the source code of the module to test
• Control flow based testing techniques
• Data flow based testing techniques
• Path testing
![Page 29: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/29.jpg)
Control flow based testing
• Given source code of a module, transform it to a control flowgraph
• Nodes and edges – decision nodes, joining nodes
![Page 30: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/30.jpg)
![Page 31: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/31.jpg)
![Page 32: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/32.jpg)
![Page 33: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/33.jpg)
Example: digitCount
1 int digitCount(int n)
2 {
3 int count = 0;
4 while (n != 0)
5 { n = n / 10;
6 count++;
7 }
8 return count;
9 }
![Page 34: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/34.jpg)
OOP Development 50
White-box testing
• Statement coverage
• Graph based
– Branch coverage
– Condition coverage
– Path coverage
![Page 35: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/35.jpg)
OOP Development 51
Example: Statement Coverage
1 int maxPositive(int n1, int n2) {
2 if (n1<=1 && n2<=1)
3 n1=1;
4 if (n1<=n2)
5 n1=n2;
6 return n1; }
TC Inputs: n1=n2=1
![Page 36: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/36.jpg)
OOP Development 52
Example: Branch Coverage
1 int maxPositive(int n1, int n2) {
2 if (n1<=1 && n2<=1)
3 n1=1;
4 if (n1<=n2)
5 n1=n2;
6 return n1; }
1,2
3
4
5
6
TC#2 Inputs: n1=2, n2=0
TC#1 Inputs: n1=n2=1
![Page 37: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/37.jpg)
OOP Development 53
Example: Condition Coverage
1 int maxPositive(int n1, int n2) {
2 if (n1<=1 && n2<=1)
3 n1=1;
4 if (n1<=n2)
5 n1=n2;
6 return n1; }
TC#1 Inputs: n1=n2=1
TC#2 Inputs: n1=2, n2=0
1,2
3
4
5
6
TC#3 Inputs: n1=0, n2=2
![Page 38: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/38.jpg)
OOP Development 54
Example: Path Coverage
1 int maxPositive(int n1, int n2) {
2 if (n1<=1 && n2<=1)
3 n1=1;
4 if (n1<=n2)
5 n1=n2;
6 return n1; }
TC#1 Inputs: n1=n2=1
TC#2 Inputs: n1=2, n2=0
TC#3 Inputs: n1=0, n2=2
1,2
3
4
5
6
TC#4 Inputs: n1=n2=0
![Page 39: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/39.jpg)
Path expression
• Representing in a generic way an execution sequence
• Can represent the complete program
• Use operators: + for choice
. For a concatenation
{} * for zero or more repetition
{}+ for one or more repetition
() for grouping
![Page 40: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/40.jpg)
![Page 41: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/41.jpg)
Path expression
• Useful to study properties of test sequences
• Annotate the transitions with desirable property then analyze the resulting expression
• For example: assign a probability to each transition – allows you to evaluate the probability of each path; helps you decide on testing the most probable paths
![Page 42: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/42.jpg)
Integration testing
• After testing all modules individually using both white and black box testing techniques
• Top-down integration
• Bottom-up integration
• Sandwich / hybrid integration
![Page 43: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/43.jpg)
Top down integration
• Start with top level modules and integrate left to rigth
• Good for testing GUI heavy modules
• Increase of confidence to see GUI at the beginning
• Less time to test low level utility modules – high reusability modules
• Variation: integrate i/o modules to get real input and outputs as early as possible
• Need stubs – fake modules to represent called modules
![Page 44: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/44.jpg)
![Page 45: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/45.jpg)
Bottom up integration
• Start from low level modules
• Good testing of low level utility modules
• Can see the GUI late in the process
• Require driver modules – to call the integrated modules
![Page 46: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/46.jpg)
![Page 47: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/47.jpg)
Sandwich integration testing
• Two teams – one starting bottom up and the other starting top down
• They meet in the middle
• Good testing of both high level GUI modules and low level utility modules
• Requires coordination and good management to ensure middle level modules also get a good share of the testing time
![Page 48: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/48.jpg)
OOP Development 64
System Testing
• Performed in terms of Inputs and outputs of the system.
• Performed on the targeted platform.
• Goals:
– Reveal bugs that are represented only at system level.
– Demonstrate that the system implements all required capabilities.
– Answer the question: Can we ship it yet?
![Page 49: Testing - Ku 8 Jehad Final.pdf · Test case design vs execution • Test case is developed first – verdict and remarks are empty • Test case is executed when the actual testing](https://reader033.vdocuments.site/reader033/viewer/2022051909/5ffd47a17befe51b842753db/html5/thumbnails/49.jpg)
OOP Development 65
Regression Testing
• Ensuring that modifications have not caused unintended effects.
• Retesting is required