software testing techniques software testing techniques statistical testing presentation on the...

21
Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

Upload: melvyn-shepherd

Post on 17-Jan-2016

225 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

Software testing techniquesSoftware testing techniques

Statistical Testing

Presentation on the seminar

Kaunas University of Technology

Page 2: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Introduction NIST* developed techniques for applying

statistical methods to derive best sample test cases and provide a statistical level of confidence or probability that a program implements its functional specification correctly

NIST* - National Institute of Standards and Technology

Page 3: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Introduction

The concept of statistical testing was invented by Harlan Mills (IBM)

The central idea is to use software testing as a means to assess the reliability of software as opposed to a debugging mechanism

Page 4: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Black-Box Conformance Testing

Consists of developing a suite of tests to determine the correctness of an implementation that is built to the requirements of a functional specification.

Page 5: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Black-Box Conformance Testing

Control-Flow Testing category - the objective is to model the functional specification as a directed graph and find input values for a minimal number of test cases to ensure that every achievable path is traversed by at least one of the test cases.

Page 6: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Markov chain model

It allows test input sequences to be generated from multiple probability distributions, making it more general than many existing techniques

Page 7: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Markov chain model

Analytical results associated with Markov chains facilitate informative analysis of the sequences before they are generated, indicating how the test is likely to unfold.

Page 8: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usability testing

Forecast Analyze Compare Change UI Analyze Compare …...

Page 9: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

Characterizes the population of intended uses of the software in the intended environment

Statistical testing based on a software usage model ensures that the failures that will occur most frequently in operational use will be found early in the testing cycle.

Page 10: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

Based on the software specification Can be developed in parallel with the software

Page 11: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

Improves the specification, gives an analytical description of the specification, quantifies the testing costs and, with statistical testing, provides a basis from which inferences of software reliability may be made.

Page 12: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

• 1. Review and clarify the software specification• 2. Identify expected users of the software,

expected uses of the software, and expected system environment

• 3. Define a stratification of user and environment parameters

Page 13: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

• 4. Determine the desired levels of usage model granularity

• 5. Iteratively develop usage model structure• 6. Verify the correctness of the structure against

the specification

Page 14: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Usage model

• 7. Iteratively develop a probability distribution for the model

• 8. Verify the correctness of the probability distribution against any information available concerning intended usage of the software

Page 15: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Metrics(reliability)

Probability of failure on demand Time between failures Failures count over some period of time

Page 16: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Metrics(usability)

State life duration Probability of reaching state 'X' Count of reaching state 'X' Reaching some state only once or n times Total state switches during program run-time

Page 17: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

How do we get probabilities?

Specified! No?... Other similar systems! There is none?... Based on field data (there might be no info) Assumptions about expected use (I guess that

my guess is correct!) Worst case... Just split probabilities equally

Page 18: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Binomial Trials

• The binomial model describes the number of successes in n independent trials, where each trial has probability of success p.

Page 19: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

So what is it all about? Statistical testing

Find small number of test cases to do the grand job

Find how correct it is, rather than find bugs Know who, how and where will use the soft Don't wait for too late (early testing) You know the metrics – you know the project

development state

Page 20: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Questions

•?

Page 21: Software testing techniques Software testing techniques Statistical Testing Presentation on the seminar Kaunas University of Technology

04/21/23

Thank you 4 your attention!

• Questions for audience: How does Markov Chain graph looks like? How to find states reach probabilities? How is statistical testing associated with Black

Box testing? Why would you want to know the reliability of the

soft?