Fraunhofer Institute for Open Communication Systems | Kaiserin-Augusta-Allee 31 | 10589 Berlin, Germany

From textual requirements to test models- Coping with natural language ambiguities for testing

Marc-Florian Wendland | RCIS 2013 Industrial Day | 31st May, 2013 | Paris, France

Agenda

Motivation Requirements and model-based testing Conclusion

Agenda

Motivation Requirements and model-based testing Conclusion

req. spec. design impl. in field

- bugs introduced

- bugs found

[4]

Costs per bug found

From textual requirements to test modelsSome facts about requirements

[1]

Costs of fixing software defects: • logical architecture 1.000 €

• technical architecture 4.000 €

• realisation 12.000 €

• acceptance / rollout 48.000 €

• in field 90.000 €

[2]

As much as 60% of all defects in a systems lifetime

originate from deficient requirements

[3]5%

55%

10%30%

40%

15%

45%

Test AnalysisReq. Engineering

Test Design System Dev.

From textual requirements to test modelsImpact network of requirements

RequirementsTest

Requirements

Test Case

<<verifies>>

1..*

System/Software Requirements Specification

(SRS)

System/Software

Specification

<<realizes>>

<<organizes>>

<<satisfies>>

Implementation

<<satisfies>>

System/Software Test Specification

<<realizes>>

<<covers>>

Test Exec.<<executes against>>

<<interprets>>

From textual requirements to test modelsRequirements specification techniques

Unrestricted NL Restricted NL Visual NotationFormal

language

Formalism and precision

Granted freedom to engineers

We need an approach that copes with natural language ambiguity and imprecision.

Suitable for automated transformation

Acceptance in industry

From textual requirements to test models Satisfaction with requirements engineering tasks

Requirements elicitation

Requirements analysis

Requirements specification

Requirements validation

Requirements management

high

medium

low

50% of all respondents sees unsufficiently specified requirements as biggest challenge for testing

[5]

Agenda

Motivation Requirements and model-based testing Conclusion

From textual requirements to test modelsState of the art in automated test design – Model-Based Testing

Implicit knowledge

Requirements

Formalisation

Test Plan

Test Model

TC

SUT

TC

SUTT

CSUT

• Implicit/imperfect knowledge is made explicitand (semi-)perfect

• Test design is done on the model• Repeatable, comprehensible and systematic• Prevents loss of knowledge• Model is self-documented• Quality of test model depends on experiences and

ingenuity of the tester Automated clerical task

Intellectual task

From textual requirements to test models Requirements models as starting point

We need a methodology, which ensures the requirements model is complete, unambiguous, consistent, correct and testable (IEEE 830)

Introduce a dedicated requirements model

Formalize the externally observable behavior of the system based on its requirements

Requirements model can be exploited to partially facilitate activities both system and testing side

Shared source of information for further development

From textual requirements to test modelsWhat is Behavior Engineering?

Requirements Engineering (RE)

• Requirements Elicitation• Requirements Identification• Requirements Specification

Behavior Engineering (BE)

• Requirements Formalisation• Requirements Integration• Requirements

Refinement/Completion• Requirements Validation &

Verification

Document-centric Model-centric

Phases of BE

Formalisation (RBT) Integration (pIBT) Validation (IBT)

From textual requirements to test modelsRequirements Formalisation – Translation

Requirement 1There is a single control button available for the use of the oven. If the oven door is closed and the user push the button, the oven will start cooking (that is, energize the power-tube) for one minute.

RBT

CT Translate each single requirement independently

User

From textual requirements to test modelsRequirements Integration

Precondition AxiomEvery constructive, implementable, individual functional requirement of a system, expressed as a behavior tree, has associated with it a precondition that needs to be satisfied in order for the behavior encapsulated in the functional requirement to be applicable."

Interaction AxiomFor each individual functional requirement of a system, expressed as a behavior tree, the precondition it needs to have satisfied in order to exhibit its encapsulated behavior, must be established by the behavior tree of at least one other functional requirement that belongs to the set of functional requirements of the system."

From textual requirements to test modelsRequirements Integration

Precondition AxiomEvery constructive, implementable, individual functional requirement of a system, expressed as a behavior tree, has associated with it a precondition that needs to be satisfied in order for the behavior encapsulated in the functional requirement to be applicable."

Interaction AxiomFor each individual functional requirement of a system, expressed as a behavior tree, the precondition it needs to have satisfied in order to exhibit its encapsulated behavior, must be established by the behavior tree of at least one other functional requirement that belongs to the set of functional requirements of the system."

From textual requirements to test models(Test) Analysis model as result of BE

IBT CT

Fokus!BEBE with UML

Agenda

Motivation Requirements and model-based testing Conclusion

From textual requirements to test models Conclusion

Requirements are still mostly specified textually with unrestricted natural language

Requirements validation techniques need to cope with unrestricted natural language

BE is a methodology that is aiming at validating the requirements by integration

BE alleviates the transition from a requirements model to further models

Testing is highly depending on the quality of the requirements

MBT is highly depending on the quality of the model that model the requirements

BE is predestined to tackle current model-based testing challenges

BE with UML seems wise since system/test models are often represented in UML/UTP

Thanks for your attention.Questions?

References

[1] Boehm, Berry: Software Engineering Economics, Englewood Cliffs, NJ : Prentice-Hall, 1981. ISBN 0-13-822122-7

[2] Spillner, A: SQS Empirische Daten aus 3000 IT-Projekten. 1st Testing Day Franken, Nürnberg, 10-March-2009[3] Berry, D. M.: Formal methods: the very idea - some thoughts about why they work when they work. Sci. Comput. Program., 42(1):11–27, 2002[4] Standish Group: ChAOS Reports. URL: http://www.standishgroup.com. Last visit: January 05, 2012 [5] SwissQ, Trends und Benchmark Report Schweiz, Wo stehen wir, wo geht es hin? Testing 2013, http://de.slideshare.net/swissq/testing-trends-und-benchmarks-2013-web