software testing techniques software testing techniques statistical testing presentation on the...
TRANSCRIPT
Software testing techniquesSoftware testing techniques
Statistical Testing
Presentation on the seminar
Kaunas University of Technology
04/21/23
Introduction NIST* developed techniques for applying
statistical methods to derive best sample test cases and provide a statistical level of confidence or probability that a program implements its functional specification correctly
NIST* - National Institute of Standards and Technology
04/21/23
Introduction
The concept of statistical testing was invented by Harlan Mills (IBM)
The central idea is to use software testing as a means to assess the reliability of software as opposed to a debugging mechanism
04/21/23
Black-Box Conformance Testing
Consists of developing a suite of tests to determine the correctness of an implementation that is built to the requirements of a functional specification.
04/21/23
Black-Box Conformance Testing
Control-Flow Testing category - the objective is to model the functional specification as a directed graph and find input values for a minimal number of test cases to ensure that every achievable path is traversed by at least one of the test cases.
04/21/23
Markov chain model
It allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques
04/21/23
Markov chain model
Analytical results associated with Markov chains facilitate informative analysis of the sequences before they are generated, indicating how the test is likely to unfold.
04/21/23
Usability testing
Forecast Analyze Compare Change UI Analyze Compare …...
04/21/23
Usage model
Characterizes the population of intended uses of the software in the intended environment
Statistical testing based on a software usage model ensures that the failures that will occur most frequently in operational use will be found early in the testing cycle.
04/21/23
Usage model
Based on the software specification Can be developed in parallel with the software
04/21/23
Usage model
Improves the specification, gives an analytical description of the specification, quantifies the testing costs and, with statistical testing, provides a basis from which inferences of software reliability may be made.
04/21/23
Usage model
• 1. Review and clarify the software specification• 2. Identify expected users of the software,
expected uses of the software, and expected system environment
• 3. Define a stratification of user and environment parameters
04/21/23
Usage model
• 4. Determine the desired levels of usage model granularity
• 5. Iteratively develop usage model structure• 6. Verify the correctness of the structure against
the specification
04/21/23
Usage model
• 7. Iteratively develop a probability distribution for the model
• 8. Verify the correctness of the probability distribution against any information available concerning intended usage of the software
04/21/23
Metrics(reliability)
Probability of failure on demand Time between failures Failures count over some period of time
04/21/23
Metrics(usability)
State life duration Probability of reaching state 'X' Count of reaching state 'X' Reaching some state only once or n times Total state switches during program run-time
04/21/23
How do we get probabilities?
Specified! No?... Other similar systems! There is none?... Based on field data (there might be no info) Assumptions about expected use (I guess that
my guess is correct!) Worst case... Just split probabilities equally
04/21/23
Binomial Trials
• The binomial model describes the number of successes in n independent trials, where each trial has probability of success p.
04/21/23
So what is it all about? Statistical testing
Find small number of test cases to do the grand job
Find how correct it is, rather than find bugs Know who, how and where will use the soft Don't wait for too late (early testing) You know the metrics – you know the project
development state
04/21/23
Questions
•?
04/21/23
Thank you 4 your attention!
• Questions for audience: How does Markov Chain graph looks like? How to find states reach probabilities? How is statistical testing associated with Black
Box testing? Why would you want to know the reliability of the
soft?