ingegneria del software ii - di.univaq.it filelecturer: henry muccini and vittorio cortellessa...
TRANSCRIPT
Lecturer:Henry Muccini and Vittorio Cortellessa
Computer Science Department University of L'Aquila - Italy
[email protected] – [email protected][www.di.univaq.it/muccini] – [www.di.univaq.it/cortelle]
Course:
Ingegneria del Software IIacademic year: 2004-2005
Course Web-site: [www.di.univaq.it/ingegneria2/]
Model-Checking plus Testing: from SoftwareArchitecture Analysis to Code Testing
2SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Copyright Notice
» The material in these slides may be freely reproduced and distributed, partially or totally, as far as an explicit reference or acknowledge to the material author is preserved.
Henry Muccini
3SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Acknowledgment
» This work is joined with Patrizio Pelliccione (University of L’Aquila), and Pierluigi Pierini (Siemens CNX), and Antonio Bucchiarone (ISTI – CNR)
» Published in ITM 2004, Lecture Notes in Computer Science, LNCS, vol. 3236, pp. 351 - 365 (2004).
4SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Agenda
» 1. Introduction and Motivations
» 2. Proposal
» 3. Case Study and Initial Results
123
123
12
5SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Henry Muccini: main research areasSEA Group
Software Engineering and Architecture Group
» SA-based Code Testing:
- Model-Checking driven SA-based Testing
- SA-based Code Regression Testing
- Testing of Product Line Architectures
» Model-based and Component-based Testing
» Product Line:
- Testing Product Lines
- Waple: Web Applications Product Line Engineering
123
6SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Software Model Checking and Software Testing
» Model Checking:
- It checks whether a certain property is valid for a certain modelof a system [Ruys_PhDThesis]
> Model checking is a model-based, automatic technique that, given a finite-state model M of a system and a property P, checks the validity of P in M
» Testing:
- “Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitablyselected from the usually infinite executions domain, against the specified expected behavior” [Bertolino_SWEBOK]
123
7SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Differences
123 skills on formal methods
generally not requiredskills on formal methods
test case identification problemstate explosion problem
code-based, model-based, specification-based
only model-based
clever selection of limited and relevant test cases
- usually left to the testerexperience
exhaustive approach tocompletely check the system
- completely automated
TestingModel Checking
8SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Goals and Motivations > 1/2
» General Goal:
-- integrationintegration of of modelmodel--checkingchecking and and testingtesting toprovide an useful tool to test modern complexsoftware systems
» In related approaches:
- By using model-checking features, counter-examples are produced, successively used to derive test cases
- Main Limitations:
> P1 : due to models complexity, the model checker techniques become inapplicable, thus not allowing to identify test cases;
> P2 : even on little examples, the number of generated test cases causes
123
9SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Goals and Motivations > 2/2
» Our Goal:
To apply Model-checking and Testing in a SoftwareArchitecture-based (SA) process, where:
> Model-checking techniques are used to validate the SA model conformance with respect to selected functionalproperties
+ avoiding state explosion problem
> while testing techniques are used to provide confidence on the implementation fulfillment to its architecturalspecification
+ Test case selection driven by model-checking
123
10SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Software Architecture: a bit of history
» In the early 90’s, SA is recognized as an independent discipline
» Initially boxes & arrows, informal diagrams; then, formal Architectural Description Languages (ADLs) are introduced. Recently, UML may be used to model SAs.
» Currently
- SA used for analysis purposes;
- SA as the basis for Product Family, Component-based paradigms
123
11SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Our Proposal
Charmy
validate the SA model conformance with respect to selected functional properties
provide confidence on the implementation fulfillment to its architectural spec
SA-based Testing
Requirements
CBSSoftware
Architecture
CBSImplementation
TestExec
[NOK]Fault removal
M.C.[NOK]
refine SA OK [test case selection]
drive
identify
Model
Testing
Checking
SA conformance to requirements through MC
SA model
Implementation conformance to SA through Testing
Test Caseselection
functionalproperties
TestCases
123
12SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
CHARMY (www.di.univaq.it/charmy)
123
SA spec properties
Automaticallygenerated
Step1
Step2
Step3
13SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
CHARMY» Iterative process
» Tool support:
- offers a graphical user interface to draw state diagrams and scenarios
- a plugin which allows to input existing diagram in the XMI format
- a translation engine to automatically derive Promela code and Buchi Automaton
123
14SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Component
Component
Component
Connector
SA-based Code Testing [IEEE_TSE04]
CodeSoftware Architecture spec.
XClient
ClientA ClientB
ClientC
Class--------------------------
123
15SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
SA-based Testing [IEEE TSE04, BookChapter03]
123
16SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Siemens CNX : main research areas» Siemens CNX S.p.a. is a Siemens R&D lab; its mission is the
design and development of SDH(1) TLC equipments
» relevant research areas:
- Formal design methodologies
- System and software performance analysis
- Test design methodologies
- Intelligent agent application
- Network Processors
- Ethernet first mile
- Optics and cristal properties
- Electromagnetic compatibility
1) SDH Synchronous Digital Hierarchy
123
17SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Test Design Methodology > objective
»Improve the tets design process
123
18SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > some definitions
» A SDH Network Element (NE, i.e. equipment) is modeled using the functional model standardized by ETSI and ITU-T.
» The functional model is built around two specific concepts:
- “network layering", with a client/server relationship between adjacent layers;
- “atomic functions“ (connection, termination and adaptation), to specify the behavior of each layer.
» applicative functions should reside on top of a layer providing specific processing on transmitted information
» A “virtual network connection” can be established between mate network layers (or atomic/applicative functions) belonging to different NEs by means of transport services offered by the underlying layers
123
19SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > some definitions
123
20SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > EOW architecture
123
» The EOW supports a telephone link between NEs using dedicated voice channels defined on the SDH frame (i.e. the “EOW SubNetwork” [eowSN]);
» An EOW node consists of:
- A “handset” (HS) that manage the physical phone device;
- a “conference manager” (CM) that control the handset connection to the EOW subnetwork;
eowSNHS
CM
HS
CM
HS
CM
EOWNode
21SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > EOW components
HS1
CM1
eowSN
localNumSign1
call1
callRequest
callRequest
callRequest
eowKeyDigit
eowKeyDigit
eowKeyDigit
HS2
CM2localNumSign2
call2
HS3
CM3localNumSign3
call3
123
offH
ook onH
ook?call
t imeo
ut
[cbu
sy==
true]
[cbusy==false]
timeout
onHook
!localNumSign
?call
?localNumSign/cbusy=false
[digit
==0]/
!call
22SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > Functional Requirements
» EOW Functional Requirements/Properties:
A) when an operator makes a call dialling a selective number, the target operator must receive the call.
B) it must be possible to enter a busy conference (with the special number-sign key) when a call is already in progress.
C) It must be always possible to exit to the conference (cleanly terminate a call).
123
23SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study > Functional Requirements
123
24SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Case Study ResultsInteractive simulation & Test generation
» Simulation without constraint will result in an intractable number of traces;
» Simulation conditioned by the given properties;
» Up to 36 test traces was extracted;
- Most of them are eligible to become test cases;
» Test selection focus on some optimization criteria like:
- Maximization of system coverage,
- Minimization of global number of tests
- Minimization of test lenght (i.e. number of steps)
123
25SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Some Considerations
Advantages:
» Model complexity and the state explosion reduction obtained by: SA-level model chekcing, iterative approach and abstraction ;
» Charmy → easy to use, practical, approach to model-checking, hiding the modeling complexity;
» interactive simulation → we may identify traces of interest for testing the whole system or just a relevant subsystem.
» test specifications are identified from the architectural model (not from requirements)
- Easiest alignment between SA and Test specifications;
- Easiest control of the design steps and evolution
123
26SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Some Considerations
Limitations:
» The Test Generator Engine can be automated; its implementation is in progress.
» The executable tests implementation from the generated test specifications is not automated yet. We approach this point with the aim to automate also this step.
» Models dimension and complexity still remain an issue, even if the iterative approach reduces the state explosion problem.
123
27SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Ongoing and Future work
» Improve the Simulation Process and the Test Generation Engine
» automated way to produce test cases from test specifications
» Other case studies
» Approach empirical evaluation
123
28SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Test Generation Engine
123
29SEA Group
© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group
Contact Information» Henry Muccini and Patrizio Pelliccione
- Dipartimento di Informatica, Universita' dell'Aquila, L'Aquila, Italy
- [email protected], [email protected]
» Antonio Bucchiarone
- ISTI CNR
» Pierluigi Pierini
- Siemens C.N.X. S.p.A., R. & D., L'Aquila, Italy