software engineering - chp7- tests
TRANSCRIPT
MedTech
Chapter 7 – Tests
Types of Tests, Verification, Methods…
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 1
MedTech – Mediterranean Institute of TechnologyCS321-Software Engineering
MedTech
MedTech
Why?
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 2
Software Testing
MedTech
Why?
• Because Software is Buggy!
• Cost of bugs to the US economy 60 billion dollars per year
• Software contain in average 1-5 bugs every 1000 lines of code• Having 100% correct mass-market software is impossible
• We must verify the software as much as possible
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 3
Software Testing
MedTech
Common Terms
• Official IEEE terminology• Failure
• Observable incorrect behaviour of the software
• More related to the behavior of the program rather than its code
• Fault• Also called a bug
• Incorrect code
• Necessarely but not sufficient condition of the occurrence of a failure
• Error• Cause of a fault
• Human error, conceptual…
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 4
Software Testing
MedTech
Common Terms: Example
• Let’s take this method: int doubleValue(int val){
int result;result = val * val;return result;
}
• Is the fact that this method is supposed to return the double of the input but doesn’t is considered a failure, fault or error?• Failure, because it is related to the behavior of the method
• In this case, which line of code represents the fault?• result = val * val;
• And what is the error that caused this bug?• Only God and the developer know…
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 5
Software Testing
MedTech
VERIFICATION APPROACHESSoftware Testing
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 6
MedTech
Verification Approaches
• How can we verify our software system?
• Four main approaches:• Testing (dynamic verification)
• Static verification
• Inspections
• Formal Proofs of Correctness
• In our course, we will focus on testing, as it is the most common and used verification approach• We start first by summarizing the difference between these approaches
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 7
Verification Approaches
MedTech
Testing
• Testing:• Exercising the system to try to make it fail
• Test Case: {𝑖 ∈ 𝐷, 𝑜 ∈ 𝑂}• A set of inputs in the domain D, and an expected output of the demain O
• Test Suite: Set of test cases
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 8
Verification Approaches
MedTech
Static Verification
• Static Verification:• Tries to identify precise causes of problems in the program, such as null
pointer
• Instead of considering a precise input, it considers all possible inputs (executions)
• Unlike testing, the verification is complete
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 9
Verification Approaches
MedTech
Inspections
• Inspections:• Also called Reviews or Walkthroughs
• Manual and group activity, where several developers look at the code in order to identify defects
• Quite effective in practice
• Used widely in industry
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 10
Verification Approaches
MedTech
Formal Proof
• Formal Proof of Correctness:• Given a software specification, a formal specification is a document that
formally defines the expected behaviour of the software
• A formal proof of correctness prooves that the program being verifiedactually implements the program specification
• Is done through a sofisticated mathematical analysis of the specificationand of the code
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 11
Verification Approaches
MedTech
Pros and Cons
Approach Pros Cons
Testing No False Positives Highly incomplete : testing considers a tinyfraction of the input domain
Static Verification Considers all program behaviours Considers some impossible behaviours : cangenerate false positive
Inspections SystematicThorough
Manual processInformalSubjective: depends on the people
Formal Proofs Strong guarantees Need a formal specification, rarely availableComplex and expensiveNeeds very a specialized personal
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 12
Verification Approaches
MedTech
TESTING APPROACHSoftware Testing
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 13
MedTech
Definition of Testing
• Today, Quality Assurance (Verification) is mostly about Testing• « 50% of my company employees are testers, and the rest spend 50% of
their time testing » Bill Gates• Testing
• Executing a program on a (tiny) sample of the input domain• Dynamic technique: the program has to be executed in order to test• Optimistic approximation:
• Program under test being performed on a small subset of the input domain, it istherefore supposed that the behavior of any other input is consistant with thatshown with the selected input data
• Successful tests:• « A test is successful if the program fails » Goodenough and Gerhart• Testing cannot prove the absence of error, but can only prove their presence• If all tests are passed
• Either the program is correct (unlikely)• Or the set of tests is bad (more l ikely)
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 14
Testing Approach
MedTech
Testing Granularity Levels
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 15
Testing Granularity Levels
MedTech
Testing Granularity Levels
• Unit Testing• Testing single modules one by one
• Integration Testing• Testing multiple modules• Testing their interactions
• System Testing• Testing of the complete system• Both functional and non functional requirements
• Acceptance Testing• Validation of the software against the customer requirements
• Regression Testing• Done when the system changes• Verifies that the changes did not impact the behaviour of the system
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 16
Testing Granularity Levels
MedTech
Alpha and Beta Testing
• All previous testing levels are done at the level of the developingorganization, by the developers themselves
• Alpha Testing• A form of internal acceptance testing performed mainly by inhouse software
QA and testing teams
• Last testing done by test teams at the development site• After acceptance testing
• Before releasing the software for beta tests
• Beta Testing• Final testing phase
• Testing carried out by real users in real environement• Generally more accurate because the customers are less tolerant for
problems
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 17
Testing Types
MedTech
TESTING TECHNIQUESSoftware Testing
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 18
MedTech
Black-Box and White-Box Testing
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 19
Testing Techniques
MedTech
Black-Box Testing
• A kind of testing in which we consider the software as a closed box
• Based on the description of the software (specification)• We don’t inspect the code of the software
• Objective:• Cover as much specified behavior as possible
• Limitations• Cannot reveal errors due to implementation details
• Advantages• Focuses on the domain of the software, and is used to make sure that it is
covering the important behaviors• No need for the code, can be used for legacy systems• It is possible to start test design early• Applicable at all granularity levels
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 20
Black-Box Testing
MedTech
Black-Box Testing: Example
• Specification: The function inputs an integer and prints it• Code
void printNumBytes (int param){if (param <1024) printf ("%d", param);else printf ("%d Kb",param/124);
}
• If we look only at the specification, we would perform at least 3 tests: with a positive, negative and null param• The result would be conform to the specification, if the tested integer is
less than 1024
• Black box testing would not show the error at line 3 (124 instead of 1024)
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 21
Black-Box Testing
123
MedTech
Systematic Functional Testing Approach
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 22
Black-Box Testing
MedTech
Step1: Identify Independently Testable Features
• Identifying all functionalities of the software that can be testedindependently from the others
• Example• For a simple function, there is only one testable feature
• For a complex application, like a spreadsheet, we can have multiple features:• Merge cells
• Statistical function
• Chart creation
• …
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 23
Black-Box Testing
MedTech
Step2: Select Test Data
• Identifying the relevant inputs for each function defined earlier• Consider the following case:
• How can we select inputs to run our tests?
• What are the relevant inputs, those that, once tested, can make us confident that the software is behaving correctly?
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 24
Black-Box Testing
MedTech
Step2: Select Test Data: Selection Types
• Exhaustive Testing: Test ALL possible inputs• Quite impossible in some cases, even with powerful software
• For example, for the function printSun (int a, int b), if an integer’s size is32 bits, if the system runs 1 test per nanosecond, it would take us 600 years to test all possible combinations!
• Random Testing: Pick the inputs randomly in the domain• Has advantages
• Inputs are picked uniformly: all inputs are considered equal
• Avoids designer bias
• Problem• Bugs are mostly scarse, so picking the inputs randomly will likely miss them
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 25
Black-Box Testing
MedTech
Step2: Select Test Data: Selection Types
• Partition Testing: Divide the input domaininto partitions• Partitions : areas of the domain that are
considered homogenuously by the software
• Idea: Bugs tend to be dense in somepartitions
• Objective: • Identify partitions in the input domain• Select inputs from each partition
• The parameters must be treatedindependenly to pick different partitions• Select Boundary Values in partitions
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 26
Black-Box Testing
MedTech
Step3: Test Cases Specifications
• Test Cases Specifications• Define how the values should be put together when actually testing the
system
• Combine the possible
inputs, using a cartesian
product, for example
• Eliminate meaninglesscombinations
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 27
Black-Box Testing
MedTech
Model-Based Testing
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 28
Black-Box Testing
MedTech
Model-Based Testing
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 29
Black-Box Testing
• To go from the specification to test cases, we have to construct a model• An abstract representation of the software under test
• We can use many possible models
• FSM: Finite State Machine• A possible model
MedTech
Model-Based Testing
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 30
Black-Box Testing
• To go from the specification to test cases, we have to construct a model• An abstract representation of the software under test
• We can use many possible models
• FSM: Finite State Machine• A possible model
MedTech
Model-Based Testing: Steps
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 31
Black-Box Testing
MedTech
Model-Based Testing: Example
1. Start with an informal specification
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 32
Black-Box Testing
MedTech
Model-Based Testing: Example
2. Build a model from the specification
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 33
Black-Box Testing
MedTech
Model-Based Testing: Example
3. Extract Test Cases fromthe model by trying to• Cover all the states
• Cover all the transitions
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 34
Black-Box Testing
MedTech
White-Box Testing
• Complementary to the black-box testing• Based on the code• Assumption:
• Executing the faulty statement is a necessary condition for revealing a fault
• Objective• Cover as much coded behavior as possible
• Limitations• Cannot reveal errors due to missing paths: parts of the software
specification that are not implemented• Because it is focused on the existing code, not on the specification
• Advantages• The quality of the white-box testing can be measured objectively• The test can be measured automatically
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 35
White-Box Testing
MedTech
White-Box Testing: Example
• Specification: The function inputs an integer parameter and returnshalf of its value if even, its value otherwise
• Codeint fun (int param){
int result = param/2;return result;
}
• If we look only at the code, we guess that the function is supposed to return the half of any number given, which is correctly done.• With white box testing only, we wouldn’t know that we have to test even
and odd numbers, and that each one gives a different output.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 36
White-Box Testing
123
MedTech
Test Coverage
• Assumption• Executing the faulty statement is a necessary condition for revealing a
fault
• So, in order for a test to have a chance to detect most of the bugs, itmust « cover » a big percentage of the code.
• Code coverage:• A measure (percentage) used to describe the degree to which the source
code of a program is executed when a particular test suite runs
• A program with high code coverage has more of its source code executedduring testing, which suggests it has a lower chance of containingundetected software bugs compared to a program with low code coverage.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 37
White-Box Testing
MedTech
Test Coverage Criteria
• Function Coverage• Has each function (or subroutine) in the program been called?
• Statement Coverage• Has each statement in the program been executed?
• Branch Coverage• Has each branch (also called DD-path (Decision to Decision Path)) of each
control structure (such as in if and case statements) been executed?
• Condition Coverage• Has each Boolean sub-expression evaluated both to true and false?
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 38
White-Box Testing
MedTech
Test Coverage Criteria
• Coverage Criteria are defined in terms of :• Test Requirements
• Elements, Entities in the code that we need to execute, in order to satisfy the criteria
• Result: • Test Specifications
• Descriptions of how the test should be in order to satisfy the requirements
• Test Cases• Instanciations of test specifications
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 39
White-Box Testing
MedTech
Test Coverage Criteria: Example
printSum (int a, int b){int result = a+b;if (result>0)
printcol("red", result);else if (result<0)
printcol("blue", result);}
Test Specifications: conditions on a and b in order to cover all test requirementsTest Spec #1: a+b > 0Test Spec #2: a+b < b
Examples of Test Cases:#1: ((a=[3], b=[9]), (output color=[red], output value=[12]))#2: ((a=[-5], b=[-8]), (output color=[blue], output value=[-13]))
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 40
White-Box Testing
Test Requirement 1
Test Requirement 2
MedTech
Statement Coverage
• Basic and simplest coverage criteria• Test Requirements
• All the statements in the program
• Coverage measure
• *+,-./01.2.3+4.56474.,.*4680479*+,-./016474.,.*46
printSum (int a, int b){int result = a+b;if (result>0)
printcol("red", result);else if (result<0)
printcol("blue", result);}
With TC #1, almost 71% of the statements are covered (st #1,2,3,4,7)
When adding TC #2, 100% are covered!
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 41
White-Box Testing
TC #1a=3b=9
TC #2a=-5b=-8
1234567
MedTech
Control-Flow Graph
• Convenient representation of the code when you want to reason about its structure
printSum (int a, int b){int result = a+b;if (result>0)
printcol("red", result);else if (result<0)
printcol("blue", result);}
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 42
White-Box Testing
1
2
3
5 4
6
7
TF
T
F
1234567
MedTech
Branch Coverage
• Branch: Outgoing edges from a decision point (if, loop…)• Test Requirements
• Branches in the program• Coverage Measure
• *+,-./01.2.3+4.5-/7*3:.680479 *+,-./01-/7*3:.6
printSum (int a, int b){int result = a+b;if (result>0)
printcol("red", result);else if (result<0)
printcol("blue", result);}
TC #1 covers branch #1TC #2 covers branch #2Branch #3 is never covered!
=> We add a new TC
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 43
White-Box Testing
1
2
3
5 4
6
7
TF
T
F
TC #1a=3b=9
TC #2a=-5b=-8
TC #3a = 0b = 0
1234567
MedTech
Condition Coverage
• Test Requirements• Individual conditions in the program
• Coverage Measure
• *+,-./0130*5;4;0*64:747/.-04:87*5<80479*+,-./0130*5;4;0*6
void main(){float x,y;read(x);read(y);if ((x==0)||(y>0))
y = y/x;else x = y+2;write(x);write(y);
}
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 44
White-Box Testing
3
4
7 6
8
x==0||y>01
2345678910
9
!(x==0||y>0)
MedTech
Condition Coverage
void main(){float x,y;read(x);read(y);if ((x==0)||(y>0))
y = y/x;else x = y+2;write(x);write(y);
}
Branch coverage = 100%
But Condition coverage is not complete, because we don’thave a test case where x==0 is true
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 45
White-Box Testing
3
4
7 6
8
x==0||y>0
12345678910
9
!(x==0||y>0)
TC #1x=5y=5
TC #2x=5y=-5
MedTech
Condition Coverage
void main(){float x,y;read(x);read(y);if ((x==0)||(y>0))
y = y/x;else x = y+2;write(x);write(y);
}
Condition coverage = 100% because with both TC we covered
all the possible values for both conditionsBut Branch coverage is not complete.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 46
White-Box Testing
3
4
7 6
8
x==0||y>0
12345678910
9
!(x==0||y>0)
TC #1x=5y=5
TC #3x= 0y=-5
MedTech
Test Criteria Subsumption
• We say that Branch Coverage subsumes Statement Coverage• If a test suite satisfies branch coverage, it satisfies statement coverage
too
• But Branch Coverage doesn’t subsume Condition Coverage and Condition Coverage doesn’t subsume Branch Coverage
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 47
White-Box Testing
Statement Coverage
Branch Coverage
Condition Coverage
MedTech
Branch and Condition Coverage
• Also called Decision and Condition Coverage
• Considers both Branch Coverage and Condition Coverage
• Test Requirements• Branches and individual conditions in the
program
• Coverage Measure• Both coverage measures
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 48
White-Box Testing
Statement Coverage
Branch Coverage
Condition Coverage
Branch and Condition Coverage
MedTech
Branch and Condition Coverage
void main(){float x,y;read(x);read(y);if ((x==0)||(y>0))
y = y/x;else x = y+2;write(x);write(y);
}
Condition coverage = 100% Branch coverage = 100%
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 49
White-Box Testing
3
4
7 6
8
x==0||y>0
12345678910
9
!(x==0||y>0)
TC #1x=5y=5
TC #3x= 0y=-5
TC #2x=5y=-5
MedTech
Other Criteria
• MC/DC: Modified Condition/Decision Coverage• Required for safety critical applications• Test important combinations of conditions instead of all of them• Extends branch and decision coverage with the requirement that each
condition should affect the decision outcome independently
• Path Coverage• Test requirements: all the paths in the program must be covered• Incredibly expensive, because we have virutally infinite number of TC
• Data Flow Coverage• Coverage of statements where a memory location is modified and
statements where this memory location is used
• Mutation Coverage• Evaluation of the quality of the test suite by modifying slightly the
statements in the code to see if the tests detect them
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 50
White-Box Testing
MedTech
Test Criteria Subsumption
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 51
White-Box Testing
Statement Coverage
Branch Coverage
Condition Coverage
Branch and Condition Coverage
MC/DC
Multiple Condition Coverage
Path CoverageMutation Coverage
Data-Flow Coverage
Theoritical Criteria
Practical Criteria
MedTech
NON FUNCTIONAL TESTINGSoftware Testing
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 52
MedTech
Non Functional Testing
• Testing an application from its non-functional attributes• Designed to figure out if your product will provide a good user
experience
• Types of non-functional testing:• Performance Testing
• Load Testing
• Stress Testing• Usability Testing
• Security Testing
• Portability Testing
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 53
Non Functional Testing
MedTech
Performance Testing
• Mostly used to identify any bottlenecks or performance issues rather thanfinding bugs in a software
• One of the most important and mandatory testing types• Determines how a system performs in terms of responsiveness and stability
under a particular workload• Tests:
• Speed (response time, data rendering and access..)• Capacity, stability, scalability
• Performance testing• Can be either qualitative or quantitative • Can be divided into different sub-types such as Load testing and Stress
testing.• Unless tests are conducted on the production hardware, from the same
machines the users will be using, there will always be a degree of uncertainty in the results.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 54
Non Functional Testing
MedTech
Load Testing
• Verifies application behavior under normal and peak load conditions• Measures
• Response times
• Throughput rates
• Resource-utilization levels
• Identifies your application’s breaking point, assuming that the breaking point occurs below the peak load condition
• Helps to determine• How many users the application can handle before performance is
compromised.
• How much load the hardware can handle before resource utilization limitsare exceeded.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 55
Non Functional Testing
MedTech
Stress Testing
• When the load placed on the system is raised beyond normal usage patterns to test the system's response at unusually high or peak loads
• Reveal application bugs that surface only under high load conditions• Synchronization issues
• Race conditions• Memory leak
• Provides an estimate of how far beyond the target load an application can go before causing failures and errors in addition to slowness.
• Allows you to establish application-monitoring triggers to warn of impending failures.
• Ensures that security vulnerabilities are not opened up by stressfulconditions.
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 56
Non Functional Testing
MedTech
Usability Testing
• A black-box technique • Used to identify any error(s) and improvements in the software by
observing the users through their usage and operation• Used in user-centered interaction design• Evaluates a product by testing it on users• Can be defined in terms of five factors
• Efficiency of use• Learn-ability• Memory-ability• Errors/safety• Satisfaction
• Usually involves systematic observation under controlled conditions to determine how well people can use the product
• Focuses more on the UX (User Experience) than the UI (User Interface)
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 57
Non Functional Testing
MedTech
Security Testing
• Tests if the application is vulnerable to attacks• Main security requirements
• Confidentiality
• Integrity
• Authentication• Availability
• Authorization
• Non-repudiation.
• In order to define the main security issues and possible mechanismsfor your system, you can consult the Open Web Application Security Project (OWASP: https://www.owasp.org)
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 58
Non Functional Testing
MedTech
TESTING DOCUMENTATIONSoftware Testing
Dr. L i l ia SFAXIwww. l i l iasfax i .wix .com /l i l iasfax i
S l id e 59
MedTech
Documentation
• Documentation of artifacts that should be developed before or duringthe testing of Software
• Documentation for software testing helps in estimating:• Required testing effort
• Test coverage• Requirement tracking/tracing, etc.
• Commonly used documented artifacts related to software testing :• Test Plan
• Test Scenario• Test Case
• Traceability Matrix
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 60
Documentation
MedTech
Test Plan
• Outlines:• The strategy that wil l be used to test an application• The resources that wil l be used• The test environment in which testing wil l be performed• The l imitations of the testing and the schedule of testing activities
• Responsible: Quality Assurance Team Lead • Includes:
• Introduction to the Test Plan document• Assumptions while testing the application• List of test cases included in testing the application• List of features to be tested• What sort of approach to use while testing the software• List of deliverables that need to be tested• The resources allocated for testing the application• Any risks involved during the testing process• A schedule of tasks and milestones to be achieved
• See the IEEE Test Plan Outline (given in the related documentation)
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 61
Documentation
MedTech
Test Scenario & Test Case
• Test Case• A set of input values, execution precondition, expected results and
executed post condition, developed to cover a certain test condition.
• Test Scenario• A detailed test procedure.
• Has many test cases associated with it.• Tends to make sure that end to end functionality of application under test
is working as expected
• Checks if the all business flows are working as expected
• The tester needs to put his/her foot in the end users shoes to check and perform the action as how they are using application under test
• The tester needs to consult or take help from the client, stakeholder or developers
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 62
Documentation
MedTech
Traceability Matrix
• Table that is used to trace the requirements during the SDLC• Can be used for:
• Forward tracing (i.e. from Requirements to Design or Coding) • Backward tracing (i.e. from Coding to Requirements)
• Each requirement in the RTM document is linked with its associatedtest case
• The Bug ID is also included and linked with its associated requirementsand test case.
• Goals:• Make sure that the software is developed as per the mentioned
requirements.• Helps in finding the root cause of any bug.• Helps in tracing the developed documents during different phases of SDLC
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 63
Documentation
MedTech
Traceability Matrix
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 64
Documentation
MedTech
References
Dr. L i l ia SFAXI www. l i l iasfax i .wix .com /l i l iasfax i
S l id e 65
• Software Development Process (MOOC), https://classroom.udacity.com/courses/ud805/lessons/3608718991/concepts/4404287840923#
• Software Testing, Tutorials Point, https://www.tutorialspoint.com/software_testing/software_testing_quick_guide.htm