test sequence reduction of wireless protocol conformance...

14
Research Article Test Sequence Reduction of Wireless Protocol Conformance Testing to Internet of Things Weiwei Lin , 1,2,3 Hongwei Zeng, 1 Honghao Gao , 4,5 Huaikou Miao , 1 and Xiaolin Wang 1,2 1 School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China 2 Shanghai Key Laboratory of Computer Soſtware Testing and Evaluating, Shanghai 201114, China 3 Department of Information Science and Technology, Taishan University, Taian 271000, China 4 Computing Center, Shanghai University, Shanghai 200444, China 5 Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai 200072, China Correspondence should be addressed to Honghao Gao; [email protected] Received 23 August 2018; Accepted 2 October 2018; Published 1 November 2018 Guest Editor: Lianyong Qi Copyright © 2018 Weiwei Lin et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Wireless communication protocols are indispensable in Internet of ings (IoT), which refer to rules and conventions that must be followed by both entities to complete wireless communication or service. Wireless protocol conformance testing concerns an effective way to judge whether a wireless protocol is carried out as expected. Starting from existing test sequence generation methods in conformance testing, an improved method based on overlapping by invertibility and multiple unique input/output (UIO) sequences is proposed in this paper. e method is accomplished in two steps: first, maximum-length invertibility-dependent overlapping sequences (IDOSs) are constructed, then a minimum-length rural postman tour covering the just constructed set of maximum-length IDOSs is generated and a test sequence is extracted from the tour. e soundness and effectiveness of the method are analyzed. eory and experiment show that desirable test sequences can be yielded by the proposed method to reveal violations of wireless communication protocols in IoT. 1. Introduction Wireless communication is essential and critical to Internet of ings (IoT) [1, 2]. e reliability of wireless communi- cation transmission largely depends on whether the wireless communication protocol is implemented as specified. Con- formance testing [3, 4] is widely used to check whether an implementation conforms to its specification in areas such as traditional communication protocols and reactive systems; i.e., there must be the same behavior in the implementation for any I/O behavior observed in the specification. In FSM- based conformance testing, a protocol is called a specification and is expressed as a Finite State Machine (FSM), while an implementation under test is considered to be a “black box”, the I/O behavior of which can only be observed. A test sequence is an I/O sequence such that whether an implementation conforms to the specification may be con- cluded by delivering the input sequence of the test sequence to the implementation and comparing the resulting output sequence with the output sequence in the test sequence. Test techniques based on state identification [5–10] are well-known in FSM-based conformance testing. Test tech- niques based on state identification are supposed to identify every state and verify every transition of the specification in the implementation. A state is said to be identified when a state identifier is delivered to the state. A transition is said to be verified when the end state of the transition is identified. A state identifier of a state is a nonempty set of input sequences for the state such that the set of corresponding output sequences can characterize the state. Unique input/output (UIO) sequence is a popular state identifier such that UIO- based techniques are commonly used for test sequence generation. Since wireless communication protocols can also be modeled by FSMs, FSM-based conformance testing is also applicable to wireless protocol conformance testing in Hindawi Security and Communication Networks Volume 2018, Article ID 3723691, 13 pages https://doi.org/10.1155/2018/3723691

Upload: others

Post on 02-Oct-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Research ArticleTest Sequence Reduction of Wireless Protocol ConformanceTesting to Internet of Things

Weiwei Lin 123 Hongwei Zeng1 Honghao Gao 45

Huaikou Miao 1 and Xiaolin Wang 12

1 School of Computer Engineering and Science Shanghai University Shanghai 200444 China2Shanghai Key Laboratory of Computer Software Testing and Evaluating Shanghai 201114 China3Department of Information Science and Technology Taishan University Taian 271000 China4Computing Center Shanghai University Shanghai 200444 China5Shanghai Key Laboratory of Intelligent Manufacturing and Robotics Shanghai 200072 China

Correspondence should be addressed to Honghao Gao gaohonghaoshueducn

Received 23 August 2018 Accepted 2 October 2018 Published 1 November 2018

Guest Editor Lianyong Qi

Copyright copy 2018 Weiwei Lin et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Wireless communication protocols are indispensable in Internet of Things (IoT) which refer to rules and conventions that mustbe followed by both entities to complete wireless communication or service Wireless protocol conformance testing concernsan effective way to judge whether a wireless protocol is carried out as expected Starting from existing test sequence generationmethods in conformance testing an improved method based on overlapping by invertibility and multiple unique inputoutput(UIO) sequences is proposed in this paperThemethod is accomplished in two steps firstmaximum-length invertibility-dependentoverlapping sequences (IDOSs) are constructed then a minimum-length rural postman tour covering the just constructed set ofmaximum-length IDOSs is generated and a test sequence is extracted from the tourThe soundness and effectiveness of the methodare analyzedTheory and experiment show that desirable test sequences can be yielded by the proposedmethod to reveal violationsof wireless communication protocols in IoT

1 Introduction

Wireless communication is essential and critical to Internetof Things (IoT) [1 2] The reliability of wireless communi-cation transmission largely depends on whether the wirelesscommunication protocol is implemented as specified Con-formance testing [3 4] is widely used to check whether animplementation conforms to its specification in areas suchas traditional communication protocols and reactive systemsie there must be the same behavior in the implementationfor any IO behavior observed in the specification In FSM-based conformance testing a protocol is called a specificationand is expressed as a Finite State Machine (FSM) whilean implementation under test is considered to be a ldquoblackboxrdquo the IO behavior of which can only be observedA test sequence is an IO sequence such that whether animplementation conforms to the specification may be con-cluded by delivering the input sequence of the test sequence

to the implementation and comparing the resulting outputsequence with the output sequence in the test sequence

Test techniques based on state identification [5ndash10] arewell-known in FSM-based conformance testing Test tech-niques based on state identification are supposed to identifyevery state and verify every transition of the specification inthe implementation A state is said to be identified when astate identifier is delivered to the state A transition is said tobe verified when the end state of the transition is identified Astate identifier of a state is a nonempty set of input sequencesfor the state such that the set of corresponding outputsequences can characterize the state Unique inputoutput(UIO) sequence is a popular state identifier such that UIO-based techniques are commonly used for test sequencegeneration

Since wireless communication protocols can also bemodeled by FSMs FSM-based conformance testing is alsoapplicable to wireless protocol conformance testing in

HindawiSecurity and Communication NetworksVolume 2018 Article ID 3723691 13 pageshttpsdoiorg10115520183723691

2 Security and Communication Networks

Table 1 UIO sequence of FSM corresponding to the connection and release process of Zigbee

No States UIO(s)Ex1 1199041 11989411199001Ex2 1199042 11989461199001Ex3 1199043 119894511989461199001

Ex4 119904411989421198943119900111990021198942119894411990011199003

s1

s3s2

s4

i1o1i 7

i2i4o1o3

i6o1i7

i5

i2i3o1o2

Figure 1 FSM corresponding to the connection and release processof Zigbee

IoT Zigbee is a typical wireless communication protocoland is divided into four layers physical layer (PHY)media access control lay (MAC) network layer (NWK)and application layer (APL) The connection and releaseprocess of nodes in the MAC layer is modeled by theFSM in Figure 1 and UIO sequences of every state arelisted in Table 1 On this basis test sequences basedon FSMs can be constructed using UIO sequences asstate identifiers There is a test sequence 1199041(11989411199001)1199044(1198942119894311990011199002)1199043(1198945)1199042(11989461199001)1199041(11989411199001)1199044(1198942119894411990011199003)1199041(11989411199001)1199044(1198947)1199041(11989411199001)1199044(1198942119894311990011199002)1199043(1198947)1199041(11989411199001)1199044 of the FSM in Figure 1 basedon UIO sequences in Table 1 When the input sequence ofthis test sequence is applied to an implementation every stateof the FSM in Figure 1 can be identified and every transitionof the FSM in Figure 1 can be verified in the implementationMeanwhile an output sequence can be obtained It can beconcluded whether the connection and release process ofnodes is executed as the specification specifies by comparingthe obtained output sequence with the expected one in thetest sequence

Test sequence reduction has long been an active researchtopic in FSM-based conformance testing One approach is toconvert the problem into theRural Chinese PostmanProblemfrom which a test sequence is extracted For the purpose oftransition verification every transition of an FSM is followedby an appended UIO sequence in the test sequence Bo Yanget al reduced test sequences by overlapping and multipleUIO sequences [11] Benefiting from overlapping more thanone transition may be verified by a single appended UIOsequence Hierons improved the method of UIO sequencesthrough the use of an invertibility criterion thereby achievingmore overlapping [12] ie verifying even more transitionsof an FSM with a single appended UIO sequence Howevermultiple UIO sequences which are conducive to test sequencereduction are not considered by Hierons

In order to reduce the cost of testing while assuringthe effectiveness this paper presents an improved methodto generate reduced test sequences for wireless protocolconformance testing of IoT Transitions in a test sequenceare of three kinds a copy of transitions in an FSM UIOsequences which have been appended to the test sequence forthe purpose of transition verification for the FSM and thetransitions which have been appended to the test sequenceIn the improved method invertibility is taken into accountto leverage as much overlapping as possible such that all thetransitions of an FSM can be verified with as few appendedUIO sequences as possible Rural symmetric augmentationis the main measure for test sequence generation Thereplicated transitions during the rural symmetric augmen-tation are exactly the transitions for concatenation of testsubsequences in the test sequence More options of ruralsymmetric augmentation can be supplied by multiple UIOsequences than those supplied by single UIO sequences iea sensible choice of UIO sequences can lead to a minimumnumber of replicated transitions during a rural symmetricaugmentation and a minimum number of transitions forconcatenation are achieved in the test sequence In this wayfurther reduced test sequences are obtained by the proposedmethod

The rest of the paper is organized as follows In Section 2the basic concepts and assumptions used in this paper arepresented The improved method is described with simpleexamples in Section 3 and the soundness and effectivenessof the method are discussed Experimental evaluation is setout in Section 4 Then the related work about the testing ofIoT is reviewed in Section 5 and the paper is concluded brieflyin Section 6

2 Preliminaries

In this section we introduce the definitions related to FSMsand graphs together with the assumptions necessary forFSM-based conformance testing

21 Definitions An FSM is formally defined as a 6-tuple119872 =(119868119874 119878 1199040 120575 120582) where

(i) 119868 and119874 are finite and nonempty sets of input symbolsand output symbols respectively

(ii) 119878 is a finite and nonempty set of states

(iii) 1199040 isin 119878 represents the initial state of119872

Security and Communication Networks 3

Table 2 UIO sequence of1198721

No States UIO(s)Ex1 1199041 1198860

Ex2 1199042

1198871

(1198861)(1198861)

(1198861)(1198871)

Ex3 1199043 (1198861)(1198860)

(iv) 120575 119878 times 119868 997888rarr 119878 denotes the state transition function(v) 120582 119878 times 119868 997888rarr 119874 is the output function

According to this definition when an input symbol 119894is delivered to the current state 119904 119872 moves to the state1199041015840 = 120575(119904 119894) with an output produced by 120582(119904 119894) Thetransition function and output function can be extended tofinite input sequences ie for an input symbol 119894 an inputsequence 120573 isin 119868lowast (where 119868lowast is the set of finite sequencesof input symbols) and a state 119904 120575(119904 119894120573) = 120575(120575(119904 119894) 120573) and120582(119904 119894120573) = 120582(119904 119894)120582(120575(119904 119894) 120573) where concatenation is denotedby juxtaposition

A transition 119905 is defined by a tuple (119904 1199041015840 119894119900) where 119904 isthe start state 119894 is an input of 119904 119900 = 120582(119904 119894) is the associatedoutput and 1199041015840 = 120575(119904 119894) is the end state A transition (119904 1199041015840 119894119900)is invertible if it is the unique transition ending at 1199041015840 with input119894 and output 119900

A UIO sequence of a state is an inputoutput sequencesuch that the inputoutput behavior exhibited by the stateis unique ie given a UIO sequence 120573 of a state 119904 and theinput sequence of 120573 is denoted by 120573119894119899 for any state 119904

1015840 = 119904 itis always true that 120582(119904 120573119894119899) = 120582(1199041015840 120573119894119899) A UIO sequence isirreducible if it is not a UIO sequence anymore when the end119894119900 symbol of the sequence is deleted UIO sequences in thispaper will be supposed irreducible unless otherwise specifiedTheremay bemore than one UIO sequence for a state and theUIO sequences may be of different length

An example FSM1198721 is described in Figure 2 where 119878 =1199041 1199042 1199043 119868 = 119886 119887119874 = 0 1 and 1199041 is the initial state of1198721 There are four invertible transitions (1199041 1199042 1198860) (1199042 1199042 1198861)(1199042 1199043 1198871) and (1199043 1199041 1198861) The UIO sequences for everystate are shown in Table 2

An FSM119872 can be perceived as a labeled directed graph119866 A state of 119872 is represented as a node of 119866 and there isan edge labeled with 119894119900 from node 119904 to 1199041015840 in 119866 if and only if120575(119904 119894) = 1199041015840 and120582(119904 119894) = 119900 in119872 where 119904 and 1199041015840 are the start andend node of the edgeThenumbers of incoming and outgoingedges of a node are called the in-degree and out-degree of thenode respectively If the in-degree equals out-degree at eachnode then the directed graph is symmetric Suppose that 119866 isan asymmetric directed graph if 119866lowast is a symmetric directedgraph generated from 119866 by making copies of the edges then119866lowast is a symmetric augmentation of 119866 When the total costof copies of edges is minimized 119866lowast is said to be a minimalsymmetric augmentation of 119866 Suppose that 119864 is the set ofedges of 119866 and 1198641015840 sube 119864 if 119866lowast is a symmetric directed graphgenerated from 119866 by making copies of the edges such thateach edge in 1198641015840 is included in 119866lowast at least once and the totalcost of copies of edges is minimized then 119866lowast is called a rural

s1

s3s2

a0 a1

b1a1

Figure 2 FSM1198721

symmetric augmentation of 119866 A sequence of contiguousedges 120573 = (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119896minus1 119904119896 119894119896minus1119900119896minus1)forms a path of 119866 where 1199041 and 119904119896 are the start and end nodeof the path respectively A path that starts and ends at thesame node forms a tour furthermore if the tour traversesevery edge of 119866 exactly once then it is an Euler tour A ruralpostman tour is a tour that traverses a given set of edges atleast once The Rural Chinese Postman Problem is to finda minimum-length rural postman tour for a given set ofedges FSMs and their directed graph representations are usedinterchangeably throughout this paper

22 Assumptions Given a specification FSM 119872 with 119899states the fault domain Ψ(119868) of 119872 is the set of all possibleimplementations of119872 over the input alphabet 119868 of119872 Ψ119899(119868)refers to the implementations with up to 119899 states inΨ(119868)Thispaper only focuses on implementations in Ψ119899(119868)

An FSM119872 is strongly connected if for any two distinctstates 119904 and 1199041015840 there is an input sequence 120573 that takes119872 from119904 to 1199041015840 ie forall119904 1199041015840 isin 119878 119904 = 1199041015840 exist120573 isin 119868lowast sdot (120575(119904 120573) = 1199041015840)

Two states 119904 and 1199041015840 are equivalent if for any input sequencethere are always the same output sequences from 119904 and 1199041015840An FSM 119872 without equivalent states is said to be minimalor reduced otherwise 119872 is reducible by joining equivalentstates

An FSM 119872 is deterministic if at any state for any inputthere is at most one transition leading to the next stateOtherwise119872 is nondeterministic

Only strongly connected minimal and deterministicFSMs are considered in this paper In addition it is assumedthat UIO sequences for each state of an FSM are available andare derived from successor trees in advance It is noted thatonly state transition functions in forms of 119878 times 119868 997888rarr 119878 areconsidered ie state transitions without any input in IoT areout of the scope of this paper

3 Test Sequence Reduction

31 Key Properties of the Method

311 Overlapping by Invertibility

Definition 1 (invertibility-dependent UIO sequence) If atransition (119904 1199041015840 119894119900) is invertible and 120573 is a UIO sequence of1199041015840 then (119894119900)sdot120573 is an invertibility-dependent UIO sequence of119904

4 Security and Communication Networks

The transition (1199041 1199042 1198860) of1198721 in Figure 2 is invertibleand (1198861)(1198871) is a UIO sequence of 1199042 then (1198860)(1198861)(1198871)is an invertibility-dependent UIO sequence of 1199041

Definition 2 (invertibility-dependent overlapping sequence)Given a transition sequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1 ifevery transition (119904119896 119904119896+1 119894119896119900119896)(1 ⩽ 119896 ⩽ 119895 minus 1) is verifiedby an invertibility-dependent UIO sequence (119894119896+1119900119896+1) sdot(119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) is verified by120573 then (1199041 1199042 11989411199001) (1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is aninvertibility-dependent overlapping sequence (IDOS)

There is a transition sequence 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 of1198721 in Figure 2 and aUIO sequence 1198860 of 1199041 When thelast transition (1199043 1199041 1198861) of 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041is verified by the UIO sequence 1198860 of 1199041 by workingbackward (1199042 1199043 1198871) is verified by (1198861)(1198860) (1199042 1199042 1198861)is verified by (1198871)(1198861)(1198860) and (1199041 1199042 1198860) is verifiedby (1198861)(1198871)(1198861)(1198860) ie all the other transitions exceptthe last one are verified by invertibility-dependent UIOsequences As a result 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anIDOS

For (119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 (1 ⩽ 119896 ⩽119895 minus 1) to be invertibility-dependent UIO sequences indis-pensable requirements for transitions in (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) are put forward

Theorem 3 Given a transition sequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOS if andonly if every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) is invertible

Proof The sufficiency of the condition is proved inductivelyby working backward In the base case when 119896 = 119895 (119904119895minus1 119904119895119894119895minus1119900119895minus1) is sure to be verified by an invertibility-dependentUIO sequence (119894119895119900119895) sdot 120573 since (119904119895 119904119895+1 119894119895119900119895) is invertibleIn the inductive step assume that (119904119896minus1 119904119896 119894119896minus1119900119896minus1) (3 ⩽119896 ⩽ 119895 minus 1) is verified by an invertibility-dependent UIOsequence 120574 then (119904119896minus2 119904119896minus1 119894119896minus2119900119896minus2) is definitely verifiedby an invertibility-dependent UIO sequence (119894119896minus1119900119896minus1) sdot 120574since (119904119896minus1 119904119896 119894119896minus1119900119896minus1) is invertible In this way everytransition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽ 119896 ⩽ 119895 minus 1) is backward veri-fied inductively by an invertibility-dependent UIO sequence(119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) isverified by 120573 Thus it is a sufficient condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to be anIDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Next the necessity of the condition is proved bycontradiction As shown in Figure 3 suppose that(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSwith a noninvertible transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895)ie there is another transition (1199041198961015840 119904119896+1 119894119896119900119896) endingat 119904119896+1 with the same input and output Obviously 119904119896cannot be identified by (119894119896119900119896) sdot (119894119896+1119900119896+1) (119894119895119900119895) sdot 120573ie (119904119896minus1 119904119896 119894119896minus1119900119896minus1) cannot be verified by (119894119896119900119896) sdot(119894119896+1119900119896+1) (119894119895119900119895) sdot120573This contradicts with the assumption

s1i1o1 ik-1ok-1

sk

sk

ikok

ikoksk+1

ik+1ok+1 ijoj sj+1

Figure 3 A fragment of an FSM

that (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSAccordingly it is a necessary condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to bean IDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Definition 4 (set of IDOSs) Given a set 119879 in which everysequence is an IDOS of an FSM119872 if every transition of119872is included in one and only one IDOS of 119879 then 119879 is a set ofIDOSs of119872

Of all the IDOSs from a state a maximum-length IDOSis the one contains no fewer transitions than any other onesThere is no doubt that the longer IDOSs are the moreoverlapping can be achieved and the shorter test sequenceswill be obtained For maximum overlapping this paper isonly interested in maximum-length IDOSs

The set 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is a set ofIDOSs of 1198721 in Figure 2 There is only one IDOS in theset since the only IDOS 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041already covers all the transitions of 1198721 Obviously1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is also a maximum-lengthIDOS starting from 1199041

312 Multiple UIO Sequences In the improved methodfor any maximum-length IDOS (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) the associated test subsequenceis expressed as (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) sdot 120573where 120573 is a UIO sequence of 119904119895+1 A minimum-lengthrural postman tour covering all the test subsequences issubsequently constructed by a rural symmetric augmentationand a test sequence is obtained from the tour Accordinglytransitions for concatenation of test subsequences in thetest sequence are derived from the transition replicationsduring the rural symmetric augmentation It is noted thatdifferent choice of UIO sequences may result in differentrural symmetric augmentations ie a minimum number oftransition replications can be achieved by a sensible choiceof UIO sequences during the rural symmetric augmentationThe minimum number of transition replications duringthe rural symmetric augmentation indicates the minimumnumber of transitions for concatenation of test subsequencesin the test sequence leading to a reduced test sequenceIn other words the result of using single UIO sequencescan only in best-case scenarios obtain the same lengthof minimum-length rural postman tours as that of usingmultiple UIO sequences

32 Design of the Method It is known from Theorem 3 thatnoninvertible transitions restrict the generation of IDOSs

Security and Communication Networks 5

From this point of view FSMs can be partitioned into twosubsets One is FSMs with only invertible transitions andthe other is FSMs with noninvertible transitions For FSMswith only invertible transitions if the FSMs are symmetrictest sequences can be obtained directly Otherwise the FSMsshould be augmented firstly So FSMs with only invertibletransitions can also be partitioned into two subsets Oneis symmetric FSMs with only invertible transitions and theother is asymmetric FSMs with only invertible transitionsGenerally FSMs are classified into three categories symmet-ric FSMs with only invertible transitions asymmetric FSMswith only invertible transitions and FSMs with noninvertibletransitions The improved method is described in two stepsfor each type of FSMs and the detail varies for different types

Step 1 Construct the set of maximum-length IDOSs

Step 2 With the consideration of multiple UIO sequencesgenerate a minimum-length rural postman tour covering theset of maximum-length IDOSs and extract a test sequencefrom the tour

321 Symmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) There is anEuler tour in a directed graph if and only if the directed graphis strongly connected and symmetric Under the assumptionthat all the FSMs are strongly connected there must be Eulertours for symmetric FSMs with only invertible transitionsAn Euler tour starting from and ending at the initial state isdefinitely an IDOS since there are only invertible transitionsfurthermore it is a maximum-length IDOS from the initialstate since there is no other one containing more transitionsIn other words an Euler tour of a symmetric FSM withonly invertible transitions is the only sequence in the set ofmaximum-length IDOSs

Note that an Euler tour starting from and ending at theinitial state may not be unique ie there may be nonuniquesets of maximum-length IDOSs Nonetheless all the sets ofmaximum-length IDOSs hold the following properties

(i) There is only one maximum-length IDOS covering allthe transitions of the FSM in every set of maximum-lengthIDOSs

(ii) The start state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

(iii) The end state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

Step 2 (test sequence generation) For symmetric FSMswith only invertible transitions test sequences are denotedby (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120573 where(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120573 is a minimum-lengthUIO sequence of 1199041 It can be inferred that test sequences fromdifferent sets of maximum-length IDOSs are always of thesame length when there is more than one set of maximum-length IDOSs As a result a randomly generated set of

maximum-length IDOSs will do for test sequence generationof symmetric FSMs with only invertible transitions

1198721 in Figure 2 is a symmetric FSM with only invertibletransitions and 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anEuler tour starting from and ending at 1199041 It is knownfrom Step 1 that 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is aset of maximum-length IDOSs of 1198721 A test sequence1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041(1198860)1199042 is resulted from Step2

322 Asymmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) An FSM isasymmetric which refers to the fact that there are nodeswhose out-degree does not equal in-degree It is known thatsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| where 119889119900119906119905(119904) and 119889119894119899(119904)denote the out-degree and in-degree of a node respectively[13] The notation |119864| refers to the number of edges in adirected graph It is concluded fromsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904)that sum119904isin119878minus 119889119900119906119905gt119894119899(119904) = sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) where 119878minus and 119878+ arethe sets of nodes whose out-degree outnumbers in-degreeand in-degree outnumbers out-degree respectively Corre-spondingly 119889119900119906119905gt119894119899(119904) and 119889119894119899gt119900119906119905(119904

1015840) represent the amount ofout-degree over in-degree for a node in 119878minus and the amountof in-degree over out-degree for a node in 119878+ respectivelyThe relation sum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| implies thatif a path passes through as many edges as possible exactlyonce then the path consumes as many out-degree and in-degree as possible Every path through a node takes up oneincoming edge as well as one outgoing edge so if the out-degree outnumbers in-degree at a node its outgoing edgescannot be traversed completely by paths passing throughthe node ie some of its outgoing edges must instead betraversed by paths starting from the nodeThus it is advisableto start the paths from nodes whose out-degree outnumbersin-degree if all the edges should be traversed exactly oncewith as few paths as possible Moreover it can be proved bycontradiction that the paths from nodes whose out-degreeoutnumbers in-degree must end at nodes whose in-degreeoutnumbers out-degree otherwise the paths will continue toextend

According to the above analysis maximum-length IDOSsgeneration for asymmetric FSMs with only invertible transi-tions is performed as follows start from a node 119904 isin 119878minus and godown along an outgoing edge until a node 1199041015840 isin 119878+ is reachedand all the outgoing edges of s1015840 have been traversed Thusa maximum-length IDOS from 119904 to 1199041015840 is obtained Add themaximum-length IDOS to the set ofmaximum-length IDOSsand delete the corresponding edges in the directed graphRepeat the above process until all the nodes of the directedgraph are isolated which implies that the set of maximum-length IDOSs is obtained

Similarly there may be nonunique sets of maximum-length IDOSs for asymmetric FSMs with only invertibletransitions and all the sets of maximum-length IDOSs holdthe following properties

(i) For any set of maximum-length IDOSs 119878minus is the set ofstart states for maximum-length IDOSs

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 2: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

2 Security and Communication Networks

Table 1 UIO sequence of FSM corresponding to the connection and release process of Zigbee

No States UIO(s)Ex1 1199041 11989411199001Ex2 1199042 11989461199001Ex3 1199043 119894511989461199001

Ex4 119904411989421198943119900111990021198942119894411990011199003

s1

s3s2

s4

i1o1i 7

i2i4o1o3

i6o1i7

i5

i2i3o1o2

Figure 1 FSM corresponding to the connection and release processof Zigbee

IoT Zigbee is a typical wireless communication protocoland is divided into four layers physical layer (PHY)media access control lay (MAC) network layer (NWK)and application layer (APL) The connection and releaseprocess of nodes in the MAC layer is modeled by theFSM in Figure 1 and UIO sequences of every state arelisted in Table 1 On this basis test sequences basedon FSMs can be constructed using UIO sequences asstate identifiers There is a test sequence 1199041(11989411199001)1199044(1198942119894311990011199002)1199043(1198945)1199042(11989461199001)1199041(11989411199001)1199044(1198942119894411990011199003)1199041(11989411199001)1199044(1198947)1199041(11989411199001)1199044(1198942119894311990011199002)1199043(1198947)1199041(11989411199001)1199044 of the FSM in Figure 1 basedon UIO sequences in Table 1 When the input sequence ofthis test sequence is applied to an implementation every stateof the FSM in Figure 1 can be identified and every transitionof the FSM in Figure 1 can be verified in the implementationMeanwhile an output sequence can be obtained It can beconcluded whether the connection and release process ofnodes is executed as the specification specifies by comparingthe obtained output sequence with the expected one in thetest sequence

Test sequence reduction has long been an active researchtopic in FSM-based conformance testing One approach is toconvert the problem into theRural Chinese PostmanProblemfrom which a test sequence is extracted For the purpose oftransition verification every transition of an FSM is followedby an appended UIO sequence in the test sequence Bo Yanget al reduced test sequences by overlapping and multipleUIO sequences [11] Benefiting from overlapping more thanone transition may be verified by a single appended UIOsequence Hierons improved the method of UIO sequencesthrough the use of an invertibility criterion thereby achievingmore overlapping [12] ie verifying even more transitionsof an FSM with a single appended UIO sequence Howevermultiple UIO sequences which are conducive to test sequencereduction are not considered by Hierons

In order to reduce the cost of testing while assuringthe effectiveness this paper presents an improved methodto generate reduced test sequences for wireless protocolconformance testing of IoT Transitions in a test sequenceare of three kinds a copy of transitions in an FSM UIOsequences which have been appended to the test sequence forthe purpose of transition verification for the FSM and thetransitions which have been appended to the test sequenceIn the improved method invertibility is taken into accountto leverage as much overlapping as possible such that all thetransitions of an FSM can be verified with as few appendedUIO sequences as possible Rural symmetric augmentationis the main measure for test sequence generation Thereplicated transitions during the rural symmetric augmen-tation are exactly the transitions for concatenation of testsubsequences in the test sequence More options of ruralsymmetric augmentation can be supplied by multiple UIOsequences than those supplied by single UIO sequences iea sensible choice of UIO sequences can lead to a minimumnumber of replicated transitions during a rural symmetricaugmentation and a minimum number of transitions forconcatenation are achieved in the test sequence In this wayfurther reduced test sequences are obtained by the proposedmethod

The rest of the paper is organized as follows In Section 2the basic concepts and assumptions used in this paper arepresented The improved method is described with simpleexamples in Section 3 and the soundness and effectivenessof the method are discussed Experimental evaluation is setout in Section 4 Then the related work about the testing ofIoT is reviewed in Section 5 and the paper is concluded brieflyin Section 6

2 Preliminaries

In this section we introduce the definitions related to FSMsand graphs together with the assumptions necessary forFSM-based conformance testing

21 Definitions An FSM is formally defined as a 6-tuple119872 =(119868119874 119878 1199040 120575 120582) where

(i) 119868 and119874 are finite and nonempty sets of input symbolsand output symbols respectively

(ii) 119878 is a finite and nonempty set of states

(iii) 1199040 isin 119878 represents the initial state of119872

Security and Communication Networks 3

Table 2 UIO sequence of1198721

No States UIO(s)Ex1 1199041 1198860

Ex2 1199042

1198871

(1198861)(1198861)

(1198861)(1198871)

Ex3 1199043 (1198861)(1198860)

(iv) 120575 119878 times 119868 997888rarr 119878 denotes the state transition function(v) 120582 119878 times 119868 997888rarr 119874 is the output function

According to this definition when an input symbol 119894is delivered to the current state 119904 119872 moves to the state1199041015840 = 120575(119904 119894) with an output produced by 120582(119904 119894) Thetransition function and output function can be extended tofinite input sequences ie for an input symbol 119894 an inputsequence 120573 isin 119868lowast (where 119868lowast is the set of finite sequencesof input symbols) and a state 119904 120575(119904 119894120573) = 120575(120575(119904 119894) 120573) and120582(119904 119894120573) = 120582(119904 119894)120582(120575(119904 119894) 120573) where concatenation is denotedby juxtaposition

A transition 119905 is defined by a tuple (119904 1199041015840 119894119900) where 119904 isthe start state 119894 is an input of 119904 119900 = 120582(119904 119894) is the associatedoutput and 1199041015840 = 120575(119904 119894) is the end state A transition (119904 1199041015840 119894119900)is invertible if it is the unique transition ending at 1199041015840 with input119894 and output 119900

A UIO sequence of a state is an inputoutput sequencesuch that the inputoutput behavior exhibited by the stateis unique ie given a UIO sequence 120573 of a state 119904 and theinput sequence of 120573 is denoted by 120573119894119899 for any state 119904

1015840 = 119904 itis always true that 120582(119904 120573119894119899) = 120582(1199041015840 120573119894119899) A UIO sequence isirreducible if it is not a UIO sequence anymore when the end119894119900 symbol of the sequence is deleted UIO sequences in thispaper will be supposed irreducible unless otherwise specifiedTheremay bemore than one UIO sequence for a state and theUIO sequences may be of different length

An example FSM1198721 is described in Figure 2 where 119878 =1199041 1199042 1199043 119868 = 119886 119887119874 = 0 1 and 1199041 is the initial state of1198721 There are four invertible transitions (1199041 1199042 1198860) (1199042 1199042 1198861)(1199042 1199043 1198871) and (1199043 1199041 1198861) The UIO sequences for everystate are shown in Table 2

An FSM119872 can be perceived as a labeled directed graph119866 A state of 119872 is represented as a node of 119866 and there isan edge labeled with 119894119900 from node 119904 to 1199041015840 in 119866 if and only if120575(119904 119894) = 1199041015840 and120582(119904 119894) = 119900 in119872 where 119904 and 1199041015840 are the start andend node of the edgeThenumbers of incoming and outgoingedges of a node are called the in-degree and out-degree of thenode respectively If the in-degree equals out-degree at eachnode then the directed graph is symmetric Suppose that 119866 isan asymmetric directed graph if 119866lowast is a symmetric directedgraph generated from 119866 by making copies of the edges then119866lowast is a symmetric augmentation of 119866 When the total costof copies of edges is minimized 119866lowast is said to be a minimalsymmetric augmentation of 119866 Suppose that 119864 is the set ofedges of 119866 and 1198641015840 sube 119864 if 119866lowast is a symmetric directed graphgenerated from 119866 by making copies of the edges such thateach edge in 1198641015840 is included in 119866lowast at least once and the totalcost of copies of edges is minimized then 119866lowast is called a rural

s1

s3s2

a0 a1

b1a1

Figure 2 FSM1198721

symmetric augmentation of 119866 A sequence of contiguousedges 120573 = (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119896minus1 119904119896 119894119896minus1119900119896minus1)forms a path of 119866 where 1199041 and 119904119896 are the start and end nodeof the path respectively A path that starts and ends at thesame node forms a tour furthermore if the tour traversesevery edge of 119866 exactly once then it is an Euler tour A ruralpostman tour is a tour that traverses a given set of edges atleast once The Rural Chinese Postman Problem is to finda minimum-length rural postman tour for a given set ofedges FSMs and their directed graph representations are usedinterchangeably throughout this paper

22 Assumptions Given a specification FSM 119872 with 119899states the fault domain Ψ(119868) of 119872 is the set of all possibleimplementations of119872 over the input alphabet 119868 of119872 Ψ119899(119868)refers to the implementations with up to 119899 states inΨ(119868)Thispaper only focuses on implementations in Ψ119899(119868)

An FSM119872 is strongly connected if for any two distinctstates 119904 and 1199041015840 there is an input sequence 120573 that takes119872 from119904 to 1199041015840 ie forall119904 1199041015840 isin 119878 119904 = 1199041015840 exist120573 isin 119868lowast sdot (120575(119904 120573) = 1199041015840)

Two states 119904 and 1199041015840 are equivalent if for any input sequencethere are always the same output sequences from 119904 and 1199041015840An FSM 119872 without equivalent states is said to be minimalor reduced otherwise 119872 is reducible by joining equivalentstates

An FSM 119872 is deterministic if at any state for any inputthere is at most one transition leading to the next stateOtherwise119872 is nondeterministic

Only strongly connected minimal and deterministicFSMs are considered in this paper In addition it is assumedthat UIO sequences for each state of an FSM are available andare derived from successor trees in advance It is noted thatonly state transition functions in forms of 119878 times 119868 997888rarr 119878 areconsidered ie state transitions without any input in IoT areout of the scope of this paper

3 Test Sequence Reduction

31 Key Properties of the Method

311 Overlapping by Invertibility

Definition 1 (invertibility-dependent UIO sequence) If atransition (119904 1199041015840 119894119900) is invertible and 120573 is a UIO sequence of1199041015840 then (119894119900)sdot120573 is an invertibility-dependent UIO sequence of119904

4 Security and Communication Networks

The transition (1199041 1199042 1198860) of1198721 in Figure 2 is invertibleand (1198861)(1198871) is a UIO sequence of 1199042 then (1198860)(1198861)(1198871)is an invertibility-dependent UIO sequence of 1199041

Definition 2 (invertibility-dependent overlapping sequence)Given a transition sequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1 ifevery transition (119904119896 119904119896+1 119894119896119900119896)(1 ⩽ 119896 ⩽ 119895 minus 1) is verifiedby an invertibility-dependent UIO sequence (119894119896+1119900119896+1) sdot(119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) is verified by120573 then (1199041 1199042 11989411199001) (1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is aninvertibility-dependent overlapping sequence (IDOS)

There is a transition sequence 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 of1198721 in Figure 2 and aUIO sequence 1198860 of 1199041 When thelast transition (1199043 1199041 1198861) of 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041is verified by the UIO sequence 1198860 of 1199041 by workingbackward (1199042 1199043 1198871) is verified by (1198861)(1198860) (1199042 1199042 1198861)is verified by (1198871)(1198861)(1198860) and (1199041 1199042 1198860) is verifiedby (1198861)(1198871)(1198861)(1198860) ie all the other transitions exceptthe last one are verified by invertibility-dependent UIOsequences As a result 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anIDOS

For (119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 (1 ⩽ 119896 ⩽119895 minus 1) to be invertibility-dependent UIO sequences indis-pensable requirements for transitions in (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) are put forward

Theorem 3 Given a transition sequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOS if andonly if every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) is invertible

Proof The sufficiency of the condition is proved inductivelyby working backward In the base case when 119896 = 119895 (119904119895minus1 119904119895119894119895minus1119900119895minus1) is sure to be verified by an invertibility-dependentUIO sequence (119894119895119900119895) sdot 120573 since (119904119895 119904119895+1 119894119895119900119895) is invertibleIn the inductive step assume that (119904119896minus1 119904119896 119894119896minus1119900119896minus1) (3 ⩽119896 ⩽ 119895 minus 1) is verified by an invertibility-dependent UIOsequence 120574 then (119904119896minus2 119904119896minus1 119894119896minus2119900119896minus2) is definitely verifiedby an invertibility-dependent UIO sequence (119894119896minus1119900119896minus1) sdot 120574since (119904119896minus1 119904119896 119894119896minus1119900119896minus1) is invertible In this way everytransition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽ 119896 ⩽ 119895 minus 1) is backward veri-fied inductively by an invertibility-dependent UIO sequence(119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) isverified by 120573 Thus it is a sufficient condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to be anIDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Next the necessity of the condition is proved bycontradiction As shown in Figure 3 suppose that(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSwith a noninvertible transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895)ie there is another transition (1199041198961015840 119904119896+1 119894119896119900119896) endingat 119904119896+1 with the same input and output Obviously 119904119896cannot be identified by (119894119896119900119896) sdot (119894119896+1119900119896+1) (119894119895119900119895) sdot 120573ie (119904119896minus1 119904119896 119894119896minus1119900119896minus1) cannot be verified by (119894119896119900119896) sdot(119894119896+1119900119896+1) (119894119895119900119895) sdot120573This contradicts with the assumption

s1i1o1 ik-1ok-1

sk

sk

ikok

ikoksk+1

ik+1ok+1 ijoj sj+1

Figure 3 A fragment of an FSM

that (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSAccordingly it is a necessary condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to bean IDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Definition 4 (set of IDOSs) Given a set 119879 in which everysequence is an IDOS of an FSM119872 if every transition of119872is included in one and only one IDOS of 119879 then 119879 is a set ofIDOSs of119872

Of all the IDOSs from a state a maximum-length IDOSis the one contains no fewer transitions than any other onesThere is no doubt that the longer IDOSs are the moreoverlapping can be achieved and the shorter test sequenceswill be obtained For maximum overlapping this paper isonly interested in maximum-length IDOSs

The set 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is a set ofIDOSs of 1198721 in Figure 2 There is only one IDOS in theset since the only IDOS 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041already covers all the transitions of 1198721 Obviously1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is also a maximum-lengthIDOS starting from 1199041

312 Multiple UIO Sequences In the improved methodfor any maximum-length IDOS (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) the associated test subsequenceis expressed as (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) sdot 120573where 120573 is a UIO sequence of 119904119895+1 A minimum-lengthrural postman tour covering all the test subsequences issubsequently constructed by a rural symmetric augmentationand a test sequence is obtained from the tour Accordinglytransitions for concatenation of test subsequences in thetest sequence are derived from the transition replicationsduring the rural symmetric augmentation It is noted thatdifferent choice of UIO sequences may result in differentrural symmetric augmentations ie a minimum number oftransition replications can be achieved by a sensible choiceof UIO sequences during the rural symmetric augmentationThe minimum number of transition replications duringthe rural symmetric augmentation indicates the minimumnumber of transitions for concatenation of test subsequencesin the test sequence leading to a reduced test sequenceIn other words the result of using single UIO sequencescan only in best-case scenarios obtain the same lengthof minimum-length rural postman tours as that of usingmultiple UIO sequences

32 Design of the Method It is known from Theorem 3 thatnoninvertible transitions restrict the generation of IDOSs

Security and Communication Networks 5

From this point of view FSMs can be partitioned into twosubsets One is FSMs with only invertible transitions andthe other is FSMs with noninvertible transitions For FSMswith only invertible transitions if the FSMs are symmetrictest sequences can be obtained directly Otherwise the FSMsshould be augmented firstly So FSMs with only invertibletransitions can also be partitioned into two subsets Oneis symmetric FSMs with only invertible transitions and theother is asymmetric FSMs with only invertible transitionsGenerally FSMs are classified into three categories symmet-ric FSMs with only invertible transitions asymmetric FSMswith only invertible transitions and FSMs with noninvertibletransitions The improved method is described in two stepsfor each type of FSMs and the detail varies for different types

Step 1 Construct the set of maximum-length IDOSs

Step 2 With the consideration of multiple UIO sequencesgenerate a minimum-length rural postman tour covering theset of maximum-length IDOSs and extract a test sequencefrom the tour

321 Symmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) There is anEuler tour in a directed graph if and only if the directed graphis strongly connected and symmetric Under the assumptionthat all the FSMs are strongly connected there must be Eulertours for symmetric FSMs with only invertible transitionsAn Euler tour starting from and ending at the initial state isdefinitely an IDOS since there are only invertible transitionsfurthermore it is a maximum-length IDOS from the initialstate since there is no other one containing more transitionsIn other words an Euler tour of a symmetric FSM withonly invertible transitions is the only sequence in the set ofmaximum-length IDOSs

Note that an Euler tour starting from and ending at theinitial state may not be unique ie there may be nonuniquesets of maximum-length IDOSs Nonetheless all the sets ofmaximum-length IDOSs hold the following properties

(i) There is only one maximum-length IDOS covering allthe transitions of the FSM in every set of maximum-lengthIDOSs

(ii) The start state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

(iii) The end state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

Step 2 (test sequence generation) For symmetric FSMswith only invertible transitions test sequences are denotedby (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120573 where(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120573 is a minimum-lengthUIO sequence of 1199041 It can be inferred that test sequences fromdifferent sets of maximum-length IDOSs are always of thesame length when there is more than one set of maximum-length IDOSs As a result a randomly generated set of

maximum-length IDOSs will do for test sequence generationof symmetric FSMs with only invertible transitions

1198721 in Figure 2 is a symmetric FSM with only invertibletransitions and 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anEuler tour starting from and ending at 1199041 It is knownfrom Step 1 that 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is aset of maximum-length IDOSs of 1198721 A test sequence1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041(1198860)1199042 is resulted from Step2

322 Asymmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) An FSM isasymmetric which refers to the fact that there are nodeswhose out-degree does not equal in-degree It is known thatsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| where 119889119900119906119905(119904) and 119889119894119899(119904)denote the out-degree and in-degree of a node respectively[13] The notation |119864| refers to the number of edges in adirected graph It is concluded fromsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904)that sum119904isin119878minus 119889119900119906119905gt119894119899(119904) = sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) where 119878minus and 119878+ arethe sets of nodes whose out-degree outnumbers in-degreeand in-degree outnumbers out-degree respectively Corre-spondingly 119889119900119906119905gt119894119899(119904) and 119889119894119899gt119900119906119905(119904

1015840) represent the amount ofout-degree over in-degree for a node in 119878minus and the amountof in-degree over out-degree for a node in 119878+ respectivelyThe relation sum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| implies thatif a path passes through as many edges as possible exactlyonce then the path consumes as many out-degree and in-degree as possible Every path through a node takes up oneincoming edge as well as one outgoing edge so if the out-degree outnumbers in-degree at a node its outgoing edgescannot be traversed completely by paths passing throughthe node ie some of its outgoing edges must instead betraversed by paths starting from the nodeThus it is advisableto start the paths from nodes whose out-degree outnumbersin-degree if all the edges should be traversed exactly oncewith as few paths as possible Moreover it can be proved bycontradiction that the paths from nodes whose out-degreeoutnumbers in-degree must end at nodes whose in-degreeoutnumbers out-degree otherwise the paths will continue toextend

According to the above analysis maximum-length IDOSsgeneration for asymmetric FSMs with only invertible transi-tions is performed as follows start from a node 119904 isin 119878minus and godown along an outgoing edge until a node 1199041015840 isin 119878+ is reachedand all the outgoing edges of s1015840 have been traversed Thusa maximum-length IDOS from 119904 to 1199041015840 is obtained Add themaximum-length IDOS to the set ofmaximum-length IDOSsand delete the corresponding edges in the directed graphRepeat the above process until all the nodes of the directedgraph are isolated which implies that the set of maximum-length IDOSs is obtained

Similarly there may be nonunique sets of maximum-length IDOSs for asymmetric FSMs with only invertibletransitions and all the sets of maximum-length IDOSs holdthe following properties

(i) For any set of maximum-length IDOSs 119878minus is the set ofstart states for maximum-length IDOSs

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 3: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 3

Table 2 UIO sequence of1198721

No States UIO(s)Ex1 1199041 1198860

Ex2 1199042

1198871

(1198861)(1198861)

(1198861)(1198871)

Ex3 1199043 (1198861)(1198860)

(iv) 120575 119878 times 119868 997888rarr 119878 denotes the state transition function(v) 120582 119878 times 119868 997888rarr 119874 is the output function

According to this definition when an input symbol 119894is delivered to the current state 119904 119872 moves to the state1199041015840 = 120575(119904 119894) with an output produced by 120582(119904 119894) Thetransition function and output function can be extended tofinite input sequences ie for an input symbol 119894 an inputsequence 120573 isin 119868lowast (where 119868lowast is the set of finite sequencesof input symbols) and a state 119904 120575(119904 119894120573) = 120575(120575(119904 119894) 120573) and120582(119904 119894120573) = 120582(119904 119894)120582(120575(119904 119894) 120573) where concatenation is denotedby juxtaposition

A transition 119905 is defined by a tuple (119904 1199041015840 119894119900) where 119904 isthe start state 119894 is an input of 119904 119900 = 120582(119904 119894) is the associatedoutput and 1199041015840 = 120575(119904 119894) is the end state A transition (119904 1199041015840 119894119900)is invertible if it is the unique transition ending at 1199041015840 with input119894 and output 119900

A UIO sequence of a state is an inputoutput sequencesuch that the inputoutput behavior exhibited by the stateis unique ie given a UIO sequence 120573 of a state 119904 and theinput sequence of 120573 is denoted by 120573119894119899 for any state 119904

1015840 = 119904 itis always true that 120582(119904 120573119894119899) = 120582(1199041015840 120573119894119899) A UIO sequence isirreducible if it is not a UIO sequence anymore when the end119894119900 symbol of the sequence is deleted UIO sequences in thispaper will be supposed irreducible unless otherwise specifiedTheremay bemore than one UIO sequence for a state and theUIO sequences may be of different length

An example FSM1198721 is described in Figure 2 where 119878 =1199041 1199042 1199043 119868 = 119886 119887119874 = 0 1 and 1199041 is the initial state of1198721 There are four invertible transitions (1199041 1199042 1198860) (1199042 1199042 1198861)(1199042 1199043 1198871) and (1199043 1199041 1198861) The UIO sequences for everystate are shown in Table 2

An FSM119872 can be perceived as a labeled directed graph119866 A state of 119872 is represented as a node of 119866 and there isan edge labeled with 119894119900 from node 119904 to 1199041015840 in 119866 if and only if120575(119904 119894) = 1199041015840 and120582(119904 119894) = 119900 in119872 where 119904 and 1199041015840 are the start andend node of the edgeThenumbers of incoming and outgoingedges of a node are called the in-degree and out-degree of thenode respectively If the in-degree equals out-degree at eachnode then the directed graph is symmetric Suppose that 119866 isan asymmetric directed graph if 119866lowast is a symmetric directedgraph generated from 119866 by making copies of the edges then119866lowast is a symmetric augmentation of 119866 When the total costof copies of edges is minimized 119866lowast is said to be a minimalsymmetric augmentation of 119866 Suppose that 119864 is the set ofedges of 119866 and 1198641015840 sube 119864 if 119866lowast is a symmetric directed graphgenerated from 119866 by making copies of the edges such thateach edge in 1198641015840 is included in 119866lowast at least once and the totalcost of copies of edges is minimized then 119866lowast is called a rural

s1

s3s2

a0 a1

b1a1

Figure 2 FSM1198721

symmetric augmentation of 119866 A sequence of contiguousedges 120573 = (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119896minus1 119904119896 119894119896minus1119900119896minus1)forms a path of 119866 where 1199041 and 119904119896 are the start and end nodeof the path respectively A path that starts and ends at thesame node forms a tour furthermore if the tour traversesevery edge of 119866 exactly once then it is an Euler tour A ruralpostman tour is a tour that traverses a given set of edges atleast once The Rural Chinese Postman Problem is to finda minimum-length rural postman tour for a given set ofedges FSMs and their directed graph representations are usedinterchangeably throughout this paper

22 Assumptions Given a specification FSM 119872 with 119899states the fault domain Ψ(119868) of 119872 is the set of all possibleimplementations of119872 over the input alphabet 119868 of119872 Ψ119899(119868)refers to the implementations with up to 119899 states inΨ(119868)Thispaper only focuses on implementations in Ψ119899(119868)

An FSM119872 is strongly connected if for any two distinctstates 119904 and 1199041015840 there is an input sequence 120573 that takes119872 from119904 to 1199041015840 ie forall119904 1199041015840 isin 119878 119904 = 1199041015840 exist120573 isin 119868lowast sdot (120575(119904 120573) = 1199041015840)

Two states 119904 and 1199041015840 are equivalent if for any input sequencethere are always the same output sequences from 119904 and 1199041015840An FSM 119872 without equivalent states is said to be minimalor reduced otherwise 119872 is reducible by joining equivalentstates

An FSM 119872 is deterministic if at any state for any inputthere is at most one transition leading to the next stateOtherwise119872 is nondeterministic

Only strongly connected minimal and deterministicFSMs are considered in this paper In addition it is assumedthat UIO sequences for each state of an FSM are available andare derived from successor trees in advance It is noted thatonly state transition functions in forms of 119878 times 119868 997888rarr 119878 areconsidered ie state transitions without any input in IoT areout of the scope of this paper

3 Test Sequence Reduction

31 Key Properties of the Method

311 Overlapping by Invertibility

Definition 1 (invertibility-dependent UIO sequence) If atransition (119904 1199041015840 119894119900) is invertible and 120573 is a UIO sequence of1199041015840 then (119894119900)sdot120573 is an invertibility-dependent UIO sequence of119904

4 Security and Communication Networks

The transition (1199041 1199042 1198860) of1198721 in Figure 2 is invertibleand (1198861)(1198871) is a UIO sequence of 1199042 then (1198860)(1198861)(1198871)is an invertibility-dependent UIO sequence of 1199041

Definition 2 (invertibility-dependent overlapping sequence)Given a transition sequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1 ifevery transition (119904119896 119904119896+1 119894119896119900119896)(1 ⩽ 119896 ⩽ 119895 minus 1) is verifiedby an invertibility-dependent UIO sequence (119894119896+1119900119896+1) sdot(119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) is verified by120573 then (1199041 1199042 11989411199001) (1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is aninvertibility-dependent overlapping sequence (IDOS)

There is a transition sequence 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 of1198721 in Figure 2 and aUIO sequence 1198860 of 1199041 When thelast transition (1199043 1199041 1198861) of 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041is verified by the UIO sequence 1198860 of 1199041 by workingbackward (1199042 1199043 1198871) is verified by (1198861)(1198860) (1199042 1199042 1198861)is verified by (1198871)(1198861)(1198860) and (1199041 1199042 1198860) is verifiedby (1198861)(1198871)(1198861)(1198860) ie all the other transitions exceptthe last one are verified by invertibility-dependent UIOsequences As a result 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anIDOS

For (119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 (1 ⩽ 119896 ⩽119895 minus 1) to be invertibility-dependent UIO sequences indis-pensable requirements for transitions in (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) are put forward

Theorem 3 Given a transition sequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOS if andonly if every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) is invertible

Proof The sufficiency of the condition is proved inductivelyby working backward In the base case when 119896 = 119895 (119904119895minus1 119904119895119894119895minus1119900119895minus1) is sure to be verified by an invertibility-dependentUIO sequence (119894119895119900119895) sdot 120573 since (119904119895 119904119895+1 119894119895119900119895) is invertibleIn the inductive step assume that (119904119896minus1 119904119896 119894119896minus1119900119896minus1) (3 ⩽119896 ⩽ 119895 minus 1) is verified by an invertibility-dependent UIOsequence 120574 then (119904119896minus2 119904119896minus1 119894119896minus2119900119896minus2) is definitely verifiedby an invertibility-dependent UIO sequence (119894119896minus1119900119896minus1) sdot 120574since (119904119896minus1 119904119896 119894119896minus1119900119896minus1) is invertible In this way everytransition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽ 119896 ⩽ 119895 minus 1) is backward veri-fied inductively by an invertibility-dependent UIO sequence(119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) isverified by 120573 Thus it is a sufficient condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to be anIDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Next the necessity of the condition is proved bycontradiction As shown in Figure 3 suppose that(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSwith a noninvertible transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895)ie there is another transition (1199041198961015840 119904119896+1 119894119896119900119896) endingat 119904119896+1 with the same input and output Obviously 119904119896cannot be identified by (119894119896119900119896) sdot (119894119896+1119900119896+1) (119894119895119900119895) sdot 120573ie (119904119896minus1 119904119896 119894119896minus1119900119896minus1) cannot be verified by (119894119896119900119896) sdot(119894119896+1119900119896+1) (119894119895119900119895) sdot120573This contradicts with the assumption

s1i1o1 ik-1ok-1

sk

sk

ikok

ikoksk+1

ik+1ok+1 ijoj sj+1

Figure 3 A fragment of an FSM

that (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSAccordingly it is a necessary condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to bean IDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Definition 4 (set of IDOSs) Given a set 119879 in which everysequence is an IDOS of an FSM119872 if every transition of119872is included in one and only one IDOS of 119879 then 119879 is a set ofIDOSs of119872

Of all the IDOSs from a state a maximum-length IDOSis the one contains no fewer transitions than any other onesThere is no doubt that the longer IDOSs are the moreoverlapping can be achieved and the shorter test sequenceswill be obtained For maximum overlapping this paper isonly interested in maximum-length IDOSs

The set 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is a set ofIDOSs of 1198721 in Figure 2 There is only one IDOS in theset since the only IDOS 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041already covers all the transitions of 1198721 Obviously1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is also a maximum-lengthIDOS starting from 1199041

312 Multiple UIO Sequences In the improved methodfor any maximum-length IDOS (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) the associated test subsequenceis expressed as (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) sdot 120573where 120573 is a UIO sequence of 119904119895+1 A minimum-lengthrural postman tour covering all the test subsequences issubsequently constructed by a rural symmetric augmentationand a test sequence is obtained from the tour Accordinglytransitions for concatenation of test subsequences in thetest sequence are derived from the transition replicationsduring the rural symmetric augmentation It is noted thatdifferent choice of UIO sequences may result in differentrural symmetric augmentations ie a minimum number oftransition replications can be achieved by a sensible choiceof UIO sequences during the rural symmetric augmentationThe minimum number of transition replications duringthe rural symmetric augmentation indicates the minimumnumber of transitions for concatenation of test subsequencesin the test sequence leading to a reduced test sequenceIn other words the result of using single UIO sequencescan only in best-case scenarios obtain the same lengthof minimum-length rural postman tours as that of usingmultiple UIO sequences

32 Design of the Method It is known from Theorem 3 thatnoninvertible transitions restrict the generation of IDOSs

Security and Communication Networks 5

From this point of view FSMs can be partitioned into twosubsets One is FSMs with only invertible transitions andthe other is FSMs with noninvertible transitions For FSMswith only invertible transitions if the FSMs are symmetrictest sequences can be obtained directly Otherwise the FSMsshould be augmented firstly So FSMs with only invertibletransitions can also be partitioned into two subsets Oneis symmetric FSMs with only invertible transitions and theother is asymmetric FSMs with only invertible transitionsGenerally FSMs are classified into three categories symmet-ric FSMs with only invertible transitions asymmetric FSMswith only invertible transitions and FSMs with noninvertibletransitions The improved method is described in two stepsfor each type of FSMs and the detail varies for different types

Step 1 Construct the set of maximum-length IDOSs

Step 2 With the consideration of multiple UIO sequencesgenerate a minimum-length rural postman tour covering theset of maximum-length IDOSs and extract a test sequencefrom the tour

321 Symmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) There is anEuler tour in a directed graph if and only if the directed graphis strongly connected and symmetric Under the assumptionthat all the FSMs are strongly connected there must be Eulertours for symmetric FSMs with only invertible transitionsAn Euler tour starting from and ending at the initial state isdefinitely an IDOS since there are only invertible transitionsfurthermore it is a maximum-length IDOS from the initialstate since there is no other one containing more transitionsIn other words an Euler tour of a symmetric FSM withonly invertible transitions is the only sequence in the set ofmaximum-length IDOSs

Note that an Euler tour starting from and ending at theinitial state may not be unique ie there may be nonuniquesets of maximum-length IDOSs Nonetheless all the sets ofmaximum-length IDOSs hold the following properties

(i) There is only one maximum-length IDOS covering allthe transitions of the FSM in every set of maximum-lengthIDOSs

(ii) The start state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

(iii) The end state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

Step 2 (test sequence generation) For symmetric FSMswith only invertible transitions test sequences are denotedby (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120573 where(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120573 is a minimum-lengthUIO sequence of 1199041 It can be inferred that test sequences fromdifferent sets of maximum-length IDOSs are always of thesame length when there is more than one set of maximum-length IDOSs As a result a randomly generated set of

maximum-length IDOSs will do for test sequence generationof symmetric FSMs with only invertible transitions

1198721 in Figure 2 is a symmetric FSM with only invertibletransitions and 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anEuler tour starting from and ending at 1199041 It is knownfrom Step 1 that 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is aset of maximum-length IDOSs of 1198721 A test sequence1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041(1198860)1199042 is resulted from Step2

322 Asymmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) An FSM isasymmetric which refers to the fact that there are nodeswhose out-degree does not equal in-degree It is known thatsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| where 119889119900119906119905(119904) and 119889119894119899(119904)denote the out-degree and in-degree of a node respectively[13] The notation |119864| refers to the number of edges in adirected graph It is concluded fromsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904)that sum119904isin119878minus 119889119900119906119905gt119894119899(119904) = sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) where 119878minus and 119878+ arethe sets of nodes whose out-degree outnumbers in-degreeand in-degree outnumbers out-degree respectively Corre-spondingly 119889119900119906119905gt119894119899(119904) and 119889119894119899gt119900119906119905(119904

1015840) represent the amount ofout-degree over in-degree for a node in 119878minus and the amountof in-degree over out-degree for a node in 119878+ respectivelyThe relation sum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| implies thatif a path passes through as many edges as possible exactlyonce then the path consumes as many out-degree and in-degree as possible Every path through a node takes up oneincoming edge as well as one outgoing edge so if the out-degree outnumbers in-degree at a node its outgoing edgescannot be traversed completely by paths passing throughthe node ie some of its outgoing edges must instead betraversed by paths starting from the nodeThus it is advisableto start the paths from nodes whose out-degree outnumbersin-degree if all the edges should be traversed exactly oncewith as few paths as possible Moreover it can be proved bycontradiction that the paths from nodes whose out-degreeoutnumbers in-degree must end at nodes whose in-degreeoutnumbers out-degree otherwise the paths will continue toextend

According to the above analysis maximum-length IDOSsgeneration for asymmetric FSMs with only invertible transi-tions is performed as follows start from a node 119904 isin 119878minus and godown along an outgoing edge until a node 1199041015840 isin 119878+ is reachedand all the outgoing edges of s1015840 have been traversed Thusa maximum-length IDOS from 119904 to 1199041015840 is obtained Add themaximum-length IDOS to the set ofmaximum-length IDOSsand delete the corresponding edges in the directed graphRepeat the above process until all the nodes of the directedgraph are isolated which implies that the set of maximum-length IDOSs is obtained

Similarly there may be nonunique sets of maximum-length IDOSs for asymmetric FSMs with only invertibletransitions and all the sets of maximum-length IDOSs holdthe following properties

(i) For any set of maximum-length IDOSs 119878minus is the set ofstart states for maximum-length IDOSs

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 4: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

4 Security and Communication Networks

The transition (1199041 1199042 1198860) of1198721 in Figure 2 is invertibleand (1198861)(1198871) is a UIO sequence of 1199042 then (1198860)(1198861)(1198871)is an invertibility-dependent UIO sequence of 1199041

Definition 2 (invertibility-dependent overlapping sequence)Given a transition sequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1 ifevery transition (119904119896 119904119896+1 119894119896119900119896)(1 ⩽ 119896 ⩽ 119895 minus 1) is verifiedby an invertibility-dependent UIO sequence (119894119896+1119900119896+1) sdot(119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) is verified by120573 then (1199041 1199042 11989411199001) (1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is aninvertibility-dependent overlapping sequence (IDOS)

There is a transition sequence 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 of1198721 in Figure 2 and aUIO sequence 1198860 of 1199041 When thelast transition (1199043 1199041 1198861) of 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041is verified by the UIO sequence 1198860 of 1199041 by workingbackward (1199042 1199043 1198871) is verified by (1198861)(1198860) (1199042 1199042 1198861)is verified by (1198871)(1198861)(1198860) and (1199041 1199042 1198860) is verifiedby (1198861)(1198871)(1198861)(1198860) ie all the other transitions exceptthe last one are verified by invertibility-dependent UIOsequences As a result 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anIDOS

For (119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 (1 ⩽ 119896 ⩽119895 minus 1) to be invertibility-dependent UIO sequences indis-pensable requirements for transitions in (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) are put forward

Theorem 3 Given a transition sequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) and a UIO sequence 120573 of 119904119895+1(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOS if andonly if every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) is invertible

Proof The sufficiency of the condition is proved inductivelyby working backward In the base case when 119896 = 119895 (119904119895minus1 119904119895119894119895minus1119900119895minus1) is sure to be verified by an invertibility-dependentUIO sequence (119894119895119900119895) sdot 120573 since (119904119895 119904119895+1 119894119895119900119895) is invertibleIn the inductive step assume that (119904119896minus1 119904119896 119894119896minus1119900119896minus1) (3 ⩽119896 ⩽ 119895 minus 1) is verified by an invertibility-dependent UIOsequence 120574 then (119904119896minus2 119904119896minus1 119894119896minus2119900119896minus2) is definitely verifiedby an invertibility-dependent UIO sequence (119894119896minus1119900119896minus1) sdot 120574since (119904119896minus1 119904119896 119894119896minus1119900119896minus1) is invertible In this way everytransition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽ 119896 ⩽ 119895 minus 1) is backward veri-fied inductively by an invertibility-dependent UIO sequence(119894119896+1119900119896+1) sdot (119894119896+2119900119896+2) (119894119895119900119895) sdot 120573 when (119904119895 119904119895+1 119894119895119900119895) isverified by 120573 Thus it is a sufficient condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to be anIDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Next the necessity of the condition is proved bycontradiction As shown in Figure 3 suppose that(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSwith a noninvertible transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895)ie there is another transition (1199041198961015840 119904119896+1 119894119896119900119896) endingat 119904119896+1 with the same input and output Obviously 119904119896cannot be identified by (119894119896119900119896) sdot (119894119896+1119900119896+1) (119894119895119900119895) sdot 120573ie (119904119896minus1 119904119896 119894119896minus1119900119896minus1) cannot be verified by (119894119896119900119896) sdot(119894119896+1119900119896+1) (119894119895119900119895) sdot120573This contradicts with the assumption

s1i1o1 ik-1ok-1

sk

sk

ikok

ikoksk+1

ik+1ok+1 ijoj sj+1

Figure 3 A fragment of an FSM

that (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) is an IDOSAccordingly it is a necessary condition for a transitionsequence (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) to bean IDOS that every transition (119904119896 119904119896+1 119894119896119900119896) (2 ⩽ 119896 ⩽ 119895) isinvertible

Definition 4 (set of IDOSs) Given a set 119879 in which everysequence is an IDOS of an FSM119872 if every transition of119872is included in one and only one IDOS of 119879 then 119879 is a set ofIDOSs of119872

Of all the IDOSs from a state a maximum-length IDOSis the one contains no fewer transitions than any other onesThere is no doubt that the longer IDOSs are the moreoverlapping can be achieved and the shorter test sequenceswill be obtained For maximum overlapping this paper isonly interested in maximum-length IDOSs

The set 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is a set ofIDOSs of 1198721 in Figure 2 There is only one IDOS in theset since the only IDOS 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041already covers all the transitions of 1198721 Obviously1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is also a maximum-lengthIDOS starting from 1199041

312 Multiple UIO Sequences In the improved methodfor any maximum-length IDOS (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895) the associated test subsequenceis expressed as (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) sdot 120573where 120573 is a UIO sequence of 119904119895+1 A minimum-lengthrural postman tour covering all the test subsequences issubsequently constructed by a rural symmetric augmentationand a test sequence is obtained from the tour Accordinglytransitions for concatenation of test subsequences in thetest sequence are derived from the transition replicationsduring the rural symmetric augmentation It is noted thatdifferent choice of UIO sequences may result in differentrural symmetric augmentations ie a minimum number oftransition replications can be achieved by a sensible choiceof UIO sequences during the rural symmetric augmentationThe minimum number of transition replications duringthe rural symmetric augmentation indicates the minimumnumber of transitions for concatenation of test subsequencesin the test sequence leading to a reduced test sequenceIn other words the result of using single UIO sequencescan only in best-case scenarios obtain the same lengthof minimum-length rural postman tours as that of usingmultiple UIO sequences

32 Design of the Method It is known from Theorem 3 thatnoninvertible transitions restrict the generation of IDOSs

Security and Communication Networks 5

From this point of view FSMs can be partitioned into twosubsets One is FSMs with only invertible transitions andthe other is FSMs with noninvertible transitions For FSMswith only invertible transitions if the FSMs are symmetrictest sequences can be obtained directly Otherwise the FSMsshould be augmented firstly So FSMs with only invertibletransitions can also be partitioned into two subsets Oneis symmetric FSMs with only invertible transitions and theother is asymmetric FSMs with only invertible transitionsGenerally FSMs are classified into three categories symmet-ric FSMs with only invertible transitions asymmetric FSMswith only invertible transitions and FSMs with noninvertibletransitions The improved method is described in two stepsfor each type of FSMs and the detail varies for different types

Step 1 Construct the set of maximum-length IDOSs

Step 2 With the consideration of multiple UIO sequencesgenerate a minimum-length rural postman tour covering theset of maximum-length IDOSs and extract a test sequencefrom the tour

321 Symmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) There is anEuler tour in a directed graph if and only if the directed graphis strongly connected and symmetric Under the assumptionthat all the FSMs are strongly connected there must be Eulertours for symmetric FSMs with only invertible transitionsAn Euler tour starting from and ending at the initial state isdefinitely an IDOS since there are only invertible transitionsfurthermore it is a maximum-length IDOS from the initialstate since there is no other one containing more transitionsIn other words an Euler tour of a symmetric FSM withonly invertible transitions is the only sequence in the set ofmaximum-length IDOSs

Note that an Euler tour starting from and ending at theinitial state may not be unique ie there may be nonuniquesets of maximum-length IDOSs Nonetheless all the sets ofmaximum-length IDOSs hold the following properties

(i) There is only one maximum-length IDOS covering allthe transitions of the FSM in every set of maximum-lengthIDOSs

(ii) The start state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

(iii) The end state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

Step 2 (test sequence generation) For symmetric FSMswith only invertible transitions test sequences are denotedby (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120573 where(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120573 is a minimum-lengthUIO sequence of 1199041 It can be inferred that test sequences fromdifferent sets of maximum-length IDOSs are always of thesame length when there is more than one set of maximum-length IDOSs As a result a randomly generated set of

maximum-length IDOSs will do for test sequence generationof symmetric FSMs with only invertible transitions

1198721 in Figure 2 is a symmetric FSM with only invertibletransitions and 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anEuler tour starting from and ending at 1199041 It is knownfrom Step 1 that 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is aset of maximum-length IDOSs of 1198721 A test sequence1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041(1198860)1199042 is resulted from Step2

322 Asymmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) An FSM isasymmetric which refers to the fact that there are nodeswhose out-degree does not equal in-degree It is known thatsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| where 119889119900119906119905(119904) and 119889119894119899(119904)denote the out-degree and in-degree of a node respectively[13] The notation |119864| refers to the number of edges in adirected graph It is concluded fromsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904)that sum119904isin119878minus 119889119900119906119905gt119894119899(119904) = sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) where 119878minus and 119878+ arethe sets of nodes whose out-degree outnumbers in-degreeand in-degree outnumbers out-degree respectively Corre-spondingly 119889119900119906119905gt119894119899(119904) and 119889119894119899gt119900119906119905(119904

1015840) represent the amount ofout-degree over in-degree for a node in 119878minus and the amountof in-degree over out-degree for a node in 119878+ respectivelyThe relation sum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| implies thatif a path passes through as many edges as possible exactlyonce then the path consumes as many out-degree and in-degree as possible Every path through a node takes up oneincoming edge as well as one outgoing edge so if the out-degree outnumbers in-degree at a node its outgoing edgescannot be traversed completely by paths passing throughthe node ie some of its outgoing edges must instead betraversed by paths starting from the nodeThus it is advisableto start the paths from nodes whose out-degree outnumbersin-degree if all the edges should be traversed exactly oncewith as few paths as possible Moreover it can be proved bycontradiction that the paths from nodes whose out-degreeoutnumbers in-degree must end at nodes whose in-degreeoutnumbers out-degree otherwise the paths will continue toextend

According to the above analysis maximum-length IDOSsgeneration for asymmetric FSMs with only invertible transi-tions is performed as follows start from a node 119904 isin 119878minus and godown along an outgoing edge until a node 1199041015840 isin 119878+ is reachedand all the outgoing edges of s1015840 have been traversed Thusa maximum-length IDOS from 119904 to 1199041015840 is obtained Add themaximum-length IDOS to the set ofmaximum-length IDOSsand delete the corresponding edges in the directed graphRepeat the above process until all the nodes of the directedgraph are isolated which implies that the set of maximum-length IDOSs is obtained

Similarly there may be nonunique sets of maximum-length IDOSs for asymmetric FSMs with only invertibletransitions and all the sets of maximum-length IDOSs holdthe following properties

(i) For any set of maximum-length IDOSs 119878minus is the set ofstart states for maximum-length IDOSs

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 5: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 5

From this point of view FSMs can be partitioned into twosubsets One is FSMs with only invertible transitions andthe other is FSMs with noninvertible transitions For FSMswith only invertible transitions if the FSMs are symmetrictest sequences can be obtained directly Otherwise the FSMsshould be augmented firstly So FSMs with only invertibletransitions can also be partitioned into two subsets Oneis symmetric FSMs with only invertible transitions and theother is asymmetric FSMs with only invertible transitionsGenerally FSMs are classified into three categories symmet-ric FSMs with only invertible transitions asymmetric FSMswith only invertible transitions and FSMs with noninvertibletransitions The improved method is described in two stepsfor each type of FSMs and the detail varies for different types

Step 1 Construct the set of maximum-length IDOSs

Step 2 With the consideration of multiple UIO sequencesgenerate a minimum-length rural postman tour covering theset of maximum-length IDOSs and extract a test sequencefrom the tour

321 Symmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) There is anEuler tour in a directed graph if and only if the directed graphis strongly connected and symmetric Under the assumptionthat all the FSMs are strongly connected there must be Eulertours for symmetric FSMs with only invertible transitionsAn Euler tour starting from and ending at the initial state isdefinitely an IDOS since there are only invertible transitionsfurthermore it is a maximum-length IDOS from the initialstate since there is no other one containing more transitionsIn other words an Euler tour of a symmetric FSM withonly invertible transitions is the only sequence in the set ofmaximum-length IDOSs

Note that an Euler tour starting from and ending at theinitial state may not be unique ie there may be nonuniquesets of maximum-length IDOSs Nonetheless all the sets ofmaximum-length IDOSs hold the following properties

(i) There is only one maximum-length IDOS covering allthe transitions of the FSM in every set of maximum-lengthIDOSs

(ii) The start state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

(iii) The end state of the maximum-length IDOS is theinitial state of the FSM in every set of maximum-lengthIDOSs

Step 2 (test sequence generation) For symmetric FSMswith only invertible transitions test sequences are denotedby (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120573 where(1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120573 is a minimum-lengthUIO sequence of 1199041 It can be inferred that test sequences fromdifferent sets of maximum-length IDOSs are always of thesame length when there is more than one set of maximum-length IDOSs As a result a randomly generated set of

maximum-length IDOSs will do for test sequence generationof symmetric FSMs with only invertible transitions

1198721 in Figure 2 is a symmetric FSM with only invertibletransitions and 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is anEuler tour starting from and ending at 1199041 It is knownfrom Step 1 that 1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041 is aset of maximum-length IDOSs of 1198721 A test sequence1199041(1198860)1199042(1198861)1199042(1198871)1199043(1198861)1199041(1198860)1199042 is resulted from Step2

322 Asymmetric FSMs with Only Invertible Transitions

Step 1 (maximum-length IDOSs generation) An FSM isasymmetric which refers to the fact that there are nodeswhose out-degree does not equal in-degree It is known thatsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| where 119889119900119906119905(119904) and 119889119894119899(119904)denote the out-degree and in-degree of a node respectively[13] The notation |119864| refers to the number of edges in adirected graph It is concluded fromsum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904)that sum119904isin119878minus 119889119900119906119905gt119894119899(119904) = sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) where 119878minus and 119878+ arethe sets of nodes whose out-degree outnumbers in-degreeand in-degree outnumbers out-degree respectively Corre-spondingly 119889119900119906119905gt119894119899(119904) and 119889119894119899gt119900119906119905(119904

1015840) represent the amount ofout-degree over in-degree for a node in 119878minus and the amountof in-degree over out-degree for a node in 119878+ respectivelyThe relation sum119904isin119878 119889119900119906119905(119904) = sum119904isin119878 119889119894119899(119904) = |119864| implies thatif a path passes through as many edges as possible exactlyonce then the path consumes as many out-degree and in-degree as possible Every path through a node takes up oneincoming edge as well as one outgoing edge so if the out-degree outnumbers in-degree at a node its outgoing edgescannot be traversed completely by paths passing throughthe node ie some of its outgoing edges must instead betraversed by paths starting from the nodeThus it is advisableto start the paths from nodes whose out-degree outnumbersin-degree if all the edges should be traversed exactly oncewith as few paths as possible Moreover it can be proved bycontradiction that the paths from nodes whose out-degreeoutnumbers in-degree must end at nodes whose in-degreeoutnumbers out-degree otherwise the paths will continue toextend

According to the above analysis maximum-length IDOSsgeneration for asymmetric FSMs with only invertible transi-tions is performed as follows start from a node 119904 isin 119878minus and godown along an outgoing edge until a node 1199041015840 isin 119878+ is reachedand all the outgoing edges of s1015840 have been traversed Thusa maximum-length IDOS from 119904 to 1199041015840 is obtained Add themaximum-length IDOS to the set ofmaximum-length IDOSsand delete the corresponding edges in the directed graphRepeat the above process until all the nodes of the directedgraph are isolated which implies that the set of maximum-length IDOSs is obtained

Similarly there may be nonunique sets of maximum-length IDOSs for asymmetric FSMs with only invertibletransitions and all the sets of maximum-length IDOSs holdthe following properties

(i) For any set of maximum-length IDOSs 119878minus is the set ofstart states for maximum-length IDOSs

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 6: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

6 Security and Communication Networks

s1 s4

s3s2

a1

a1

b1

c0

a0b0

c1

Figure 4 FSM1198722

(ii) For any set of maximum-length IDOSs 119878+ is the setof end states for maximum-length IDOSs

(iii) For any node 119904 isin 119878minus the number of maximum-lengthIDOSs starting from 119904 is 119889119900119906119905gt119894119899(119904) in every set of maximum-length IDOSs

(iv) For any node 1199041015840 isin 119878+ the number of maximum-length IDOSs ending at 1199041015840 is 119889119894119899gt119900119906119905(119904

1015840) in every set ofmaximum-length IDOSs

(v) The total number of maximum-length IDOSs issum119904isin119878minus 119889119900119906119905gt119894119899(119904) which equals sum1199041015840isin119878+ 119889119894119899gt119900119906119905(119904

1015840) in each set ofmaximum-length IDOSs

Step 2 (test sequence generation) With the set of maximum-length IDOSs the same method as that of Bo Yang et al isused to construct test sequences and the detail of the methodis described in Algorithm 1 The core of the method is therural symmetric augmentation over an FSM augmented bymaximum-length IDOSs and themultiple UIO sequences forthe end states of maximum-length IDOSs The core of thesymmetric augmentation is the states involved Accordingto the above properties for any set of maximum-lengthIDOSs there is no difference about the states involved inthe symmetric augmentation such that a random set ofmaximum-length IDOSs will do for asymmetric FSMs withonly invertible transitions when there is more than one set ofmaximum-length IDOSs1198722 in Figure 4 is an asymmetric FSMwith only invertible

transitions and its UIO sequences are shown in Table 3 Thenonunique sets of maximum-length IDOSs for1198722 are listedas follows

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)11990421199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)11990421199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042Clearly all the sets of maximum-length IDOSs

satisfy the properties in this section The augmentationof 1198722 using a random set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 isshown in Figure 5 and the associated test sequence of 1198722is 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042(1198861)1199043(1198871)1199044(1198860)1199042(1198880)1199042

323 FSMs with Noninvertible Transitions

Step 1 (maximum-length IDOSs generation) To address anFSM with noninvertible transitions in a similar way to the

Table 3 UIO sequence of1198722

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

first two types of FSMs noninvertible transitions are removedfrom the FSM and saved to a set The remainder excludingnoninvertible transitions is either an FSM or more than oneconnected component with only invertible transitions Andthus maximum-length IDOSs of the remainder excludingnoninvertible transitions are generated in the same wayas symmetric or asymmetric FSMs with only invertibletransitions Naturally it comes to the same conclusion asthe first two types of FSMs that a random set of maximum-length IDOSs will do when there are nonunique sets ofmaximum-length IDOSs Next a union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand all the noninvertible transitions is derived moreoverwhenever the start state of a maximum-length IDOS is theend state of a noninvertible transition the maximum-lengthIDOS is concatenated with the noninvertible transition If amaximum-length IDOS is an Euler tour then a node whichis the end state of a noninvertible transition is preferredto be the start state of the tour The set after all possibleconcatenations is a set ofmaximum-length IDOSs of the FSMwith noninvertible transitions

Given a union of maximum-length IDOSs of the remain-der excluding noninvertible transitions as well as all thenoninvertible transitions there may be nonunique sets ofmaximum-length IDOSs for the FSM with noninvertibletransitions because of different concatenation The propertiesof nonunique sets of maximum-length IDOSs for FSMs withnoninvertible transitions are described as follows

(i) In any set of maximum-length IDOSs for any state 119904which is the start state of a maximum-length IDOS if thereare 119895maximum-length IDOSs starting from 119904 then theremustbe 119895 maximum-length IDOSs starting from 119904 in every otherset of maximum-length IDOSs

(ii) In any set of maximum-length IDOSs for any state 119904which is the end state of a maximum-length IDOS if thereare 119896maximum-length IDOSs ending at 119904 then there must be119896 maximum-length IDOSs ending at 119904 in every other set ofmaximum-length IDOSs

Step 2 (test sequence generation) For FSMs with nonin-vertible transitions test sequence generation is in the sameway as asymmetric FSMs with only invertible transitionsie by means of rural symmetric augmentation over an FSMaugmented bymaximum-length IDOSs and themultipleUIOsequences for the end states of maximum-length IDOSs

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 7: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 7

RequireFSM119872 with 119899 states 1199041 1199042 119904119899 in which 1199041 is the initial stateSet of non-invertible transitions 119875 = 120601Set of maximum-length IDOSs 119876 = 120601Set of end states for non-invertible transitions 1198781015840 = 120601UIO sequences of119872A minimum-length UIO sequence 120574 of 1199041

EnsureReduced test sequence 120573 of119872

(1) if119872 is an 119865119878119872 with119898 non-invertible transitions then(2) for i=1 to m do(3) 119875 = 119875 cup 119905119894 where 119905119894 denotes a non-invertible transition(4) 119872 = 119872 119905119894(5) 1198781015840 = 1198781015840 cup 119890119894 where 119890119894 denotes the end state of 119905119894(6) end for(7) for each state whose out-degree gt in-degree in119872 do(8) 119876=119876 cup 120572 where 120572 is a maximum-length IDOS of 119895 transitions generated from the state(9) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119895)(10) end for(11) for each connected component with Euler tours do(12) if States in 1198781015840 can be found in an Euler tour 119905 with

119896 transitions then(13) Choose a state in 1198781015840 as the initial state of 119905(14) end if(15) 119876=119876 cup 119905(16) 119872=119872 119905119894 (1 ⩽ 119894 ⩽ 119896)(17) end for(18) 119876=119876 cup 119875(19) Concatenate non-invertible transitions and maximum-length IDOSs as long as the

formerrsquos end state is the latterrsquos start state(20) 119872 = 119872 cup119881lowast where119881lowast is a new state set(21) 119872 = 119872cup 119879 where 119879 is a new transition set(22) 119872 = 119872cup 119880 where119880 is a new transition set(23) Construct a minimum-length rural postman tour over 119876 in the augmented119872 and extract a

test sequence starting from 1199041(24) Remove the transition sequence follows the UIO sequence of the maximum-length IDOS which

is verified last in the minimum-length tour(25) else(26) if 119872 is a symmetric FSM with only invertible transitions then(27) 119876 = 119876 cup 119905 where 119905 is an Euler tour starting from and ending at 1199041(28) 120573 = 119905 sdot 120574(29) else(30) execute lines (7) through (9)(31) Execute lines (20) through (24)(32) end if(33) end if

Algorithm 1 Improved method of test sequence reduction

The core of rural symmetric augmentation is still the statesinvolved It is known from the above properties that for anyset ofmaximum-length IDOSs the states involved in the ruralsymmetric augmentation are the same such that a randomconcatenation will do

Considering 1198723 in Figure 6 with UIO sequences inTable 4 the following nonunique union of maximum-lengthIDOSs of the remainder excluding noninvertible transitionsand noninvertible transitions confirm that a random unionwill do

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198861)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044

1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198860)1199042 1199044(1198881)1199041(1198861)1199042 1199041(1198890)1199044 1199043(1198890)1199044

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 8: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

8 Security and Communication Networks

s1

s3s2

a1b1

s4

a0b0

c1t1 t2

U2

U1

V2lowast

Figure 5 Augmentation of1198722

a1

a1

b1

a0b0

c1s1

s3s2c0

s4

d0

d0

Figure 6 FSM1198723

Table 4 UIO sequence of1198723

No States UIO(s)

Ex1 1199041

1198870

(1198861)(1198880)

(1198861)(1198861)

Ex2 1199042(1198861)(1198871)

1198880

Ex3 1199043 1198871

Ex4 11990441198860

1198881

Take a random union of maximum-length IDOSs ofthe remainder excluding noninvertible transitions andnoninvertible transitions 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199044(1198860)1199042 1199041(1198890)1199044 1199043(1198890)1199044 nonunique setsof maximum-length IDOSs from different concatenations1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 and 1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199041(1198890)1199044(1198860)1199042 1199043(1198890)1199044 confirm that a randomconcatenation will do The augmentation of 1198723 using arandomly concatenated set of maximum-length IDOSs1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881)1199041(1198870)1199042 1199043(1198890)1199044(1198860)1199042 1199041(1198890)1199044 is shown in Figure 7 and the resultingtest sequence is 1199041(1198890)1199044(1198881)1199041(1198861)1199042(1198880)1199042(1198861)1199043(1198871)1199044(1198881) 1199041(1198870)1199042(1198880)1199042(1198861)1199043(1198890)1199044(1198860)1199042(1198880)1199042

s1

s3s2

s4

t2

t1

U2

U1

U4

U3

a1

a1 b1a0

b0

c1

t3

c0

d0

d0

V2lowastV4

lowast

Figure 7 Augmentation of1198723

Algorithm 1 describes the detail of the improved methodfor all types of FSMs Note that self-loops are always givenpriority to traverse in the process of maximum-length IDOSsgeneration When creating the new state set 119881lowast in119872 for theend state of every maximum-length IDOS there is a statein 119881lowast When creating the new transition set 119879 in 119872 forevery maximum-length IDOS there is a transition from thestart state of the maximum-length IDOS to state V119894

lowast labeledwith the corresponding label of the maximum-length IDOSWhen creating a new transition set 119880 in 119872 for every UIOsequence of the end state of every maximum-length IDOSthere is a transition from V119894

lowast to the end state of every UIOsequence labeled with the corresponding UIO sequence

Theorem 5 Given an FSM 119872 suppose that 119876 is a set ofmaximum-length IDOSs and 120573 is a sequence over119876 generatedfrom Algorithm 1 then 120573 is a reduced test sequence of119872

Proof The soundness of 120573 is first proved ie 120573 is a testsequence of119872Then the effectiveness of120573 is assessed in termsof test generation and test execution cost respectively From ageneral standpoint the notion of cost in the context of testingis complex and can be related tomany factors In our contextthe effort required for generating test sequences is measuredin terms of test sequence computational complexity Testsequence execution cost is measured by the length of testsequences Although these are clearly approximation meth-ods for practical reasons suchmethods have been commonlyused in a number of testing studies [14ndash16]

(1) Soundness Analysis

(i) Check whether every transition defined in 119872 isverified in the implementation

All the maximum-length IDOSs in 119876 are included in120573 since 120573 is a sequence over 119876 Algorithm 1 indicates thatevery maximum-length IDOS in 120573 is followed by a UIOsequence that verifies the last transition of the sequenceAccording to Definition 2 ie for any maximum-lengthIDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 119904119895+1 119894119895119900119895) if (119904119895 119904119895+1119894119895119900119895) is verified then every transition (119904119896 119904119896+1 119894119896119900119896) (1 ⩽119896 ⩽ 119895 minus 1) is verified it is inferred that transitions of all the

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 9: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 9

maximum-length IDOSs in 119876 are verified in 120573 Accordingto Definition 4 ie every transition in119872 is included in oneand only one IDOS of119876 it is concluded that every transitiondefined in119872 is verified in the implementation by 120573

(ii) Check whether every state in 119872 is defined in theimplementation119872 is supposed to be strongly connected such that for any

state 119904 of 119872 there is at least one transition ending at 119904 It isproved that every transition defined in119872 is verified in 120573 byidentifying the end state of the transition ie every state of119872 is checked in the implementation by 120573

(2) Effectiveness Analysis(i) Analysis of Computational Complexity The test sequence120573 has the same computational complexity as those of Aho etal and Hierons since all the three test sequence generationmethods are based on the max flowmin cost problem andthe networks used in every method are of the same order(ii) Analysis of Length Reduction As mentioned above tran-sitions in a test sequence are of three kinds and the cost ofa test sequence comes from the UIO sequences which havebeen appended for the purpose of transition verification andthe transitions which have been appended for the purpose ofconcatenation of test subsequences to generate a minimum-length rural postman tour

For symmetric FSMs with only invertible transitions 120573is in the form of (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) sdot 120574where (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895 1199041 119894119895119900119895) is an Euler tourstaring from and ending at 1199041 and 120574 is a minimum-lengthUIO sequence of 1199041 ie the cost of 120573 only comes from oneappended minimum-length UIO sequence of 1199041

For asymmetric FSMs with only invertible transitions allthe transitions are divided into a least number of maximum-length IDOSs ie all the transitions are verified by a leastnumber of appended UIO sequences such that the costof 120573 from the appended UIO sequences is minimal Forany maximum-length IDOS (1199041 1199042 11989411199001)(1199042 1199043 11989421199002) (119904119895119904119895+1 119894119895119900119895) there is a test subsequence (1199041 1199042 11989411199001)(1199042 119904311989421199002) (119904119895 119904119895+1 119894119895119900119895)sdot120574 where 120574 is a UIO sequence of 119904119895+1120573 is obtained from a minimum-length rural postman tourcovering all the test subsequences The minimum-lengthrural postman tour is generated by the rural symmetricaugmentation and thus the cost of 120573 for concatenation comesfrom the replicated transitions during the rural symmetricaugmentation Benefiting from multiple UIO sequences aminimum number of transition replications is reached by asensible choice of UIO sequences during the rural symmetricaugmentation ie the cost of 120573 from the transitions forconcatenation is minimal

For FSMs with noninvertible transitions transitionsexcluding noninvertible ones are divided into a least numberof IDOSs According to Theorem 3 the acquired IDOSsconcatenate with noninvertible transitions as much as pos-sible such that a set of maximum-length IDOSs with aleast number of maximum-length IDOSs is obtained ieall the transitions of an FSM with noninvertible transitionsare verified by a least number of appended UIO sequencessuch that the cost of 120573 from the appended UIO sequences

is minimal Similarly a minimum number of transitionreplications during the rural symmetric augmentation isreached by a sensible choice of UIO sequences ie the costof 120573 from the transitions for concatenation is minimal

In short for all types of FSMs the execution cost isreduced effectively without increasing the generation costsuch that 120573 is a reduced test sequence of119872

4 Case Study

We experiment with the aforementioned 1198721 1198722 and 1198723which are random generation of different types of FSMs1198724and 1198725 are also randomly generated FSMs moreover theyare example FSMs used by Bo Yang et al andHierons respec-tively While the experimental results shed some light onhow the improved method behaves with randomly generatedFSMs they bring no insight on the test sequence reductionof FSMs that are produced by software designers For thisreason experiments on FSMsmodeling INRES protocol [17]GUI for password modification in a property managementsystem [18] page function of Gmail system [19] and theconnection and release process of MAC layer in Zigbeeprotocol are carried out and the FSMmodeling page functionof Gmail system is adjusted to satisfy the strong connectivityand UIO availability assumptions

The FSMs used in the experiment are admittedly smallHowever it is important to note that FSMs are mostly usedto model the behavior of complex classes or class clustersparticularly complex control classes in protocols or reactivesystems They are rarely used to model entire systems whichwill result in large and unmanageable models for softwareengineers and testers Completeness degree is a general factorto reveal the complexity of both large and small FSMs suchthat test sequence reduction of large FSMs can to someextent be learned through relatively small FSMswith the samedistribution of completeness degree Given an FSM with 119909inputs 119910 transitions and 119899 states completeness degree of119872 isdenoted by 119910(119909 times 119899) moreover the higher the completenessdegree is the more complex the FSM is In this sectioncompleteness degrees of FSMs range from 024 to 1

The associated data of the experiment is illustrated inTable 5 |119868119899119901119906119905| |119878119905119886119905119890| and |119879119903119886119899119904119894119905119894119900119899| denote the numberof inputs states and transitions of every FSM Completenessdegree of every FSM is calculated and listed 119879119865119874119879119878+119872119880119868119874119879119894119899V and 119879119894119899V+119872119880119868119874 are test sequences resulting from BoYang et al Hierons and the improved method respectively|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| and |119879119894119899V+119872119880119868119874| represent the length ofthe corresponding test sequences

For symmetric FSMs with only invertible transitions 119879119894119899Vand 119879119894119899V+119872119880119868119874 are both Euler tours starting from the initialstate followed by a minimum-length UIO sequence of theinitial state Thus |119879119894119899V| and |119879119894119899V+119872119880119868119874| are always the samefor symmetric FSMs with only invertible transitions Whenthe Euler tour conforms to the definition of fully overlappingtransition sequences (FOTSs) [11] |119879119865119874119879119878+119872119880119868119874| is the sameas |119879119894119899V| and |119879119894119899V+119872119880119868119874| otherwise |119879119865119874119879119878+119872119880119868119874| tends to belonger because of more appended UIO sequences as well asthe possible extra transitions for concatenation

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 10: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

10 Security and Communication Networks

Table 5 Experimental objects and associated data

FSMs Information of FSMs Length of Test Sequences1003816100381610038161003816119868119899119901119906119905

1003816100381610038161003816 |119878119905119886119905119890| |119879119903119886119899119904119894119905119894119900119899| 119862119900119898119901119897119890119905119890119899119890119904119904 119863119890119892119903119890119890 |119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

1198721 2 3 4 067 8 5 51198722 3 4 7 058 10 12 101198723 4 4 9 056 15 15 131198724 3 5 11 073 17 17 161198725 2 5 10 1 33 21 20119868119873119877119864119878 5 4 16 08 13 12 12119866119880119868 7 8 25 045 74 31 31119866119898119886119894119897 9 7 15 024 43 27 27119885119894119892119887119890119890 7 4 7 025 15 14 14

Table 6 Single sample K-S check

|119879119865119874119879119878+119872119880119868119874| |119879119894119899V| |119879119894119899V+119872119880119868119874|

119873 9 9 9Mean Value 25333 17111 16444Standard Deviation 21535 8054 8263119870119900119897119898119900119892119900119903119900V minus 119878119898119894119903119899119900V 119885 952 517 564119860119904119910119898119901119900119905119894119888 119878119894119892119899119894119891119894119888119886119899119888119890 (2 minus 119905119886119894119897119890119889) 325 952 908

Table 7 Paired samples statistics

Pair Mean N Standard Deviation Standard ErrorMean

|119879119865119874119879119878+119872119880119868119874| 25333 9 21535 7178|119879119894119899V+119872119880119868119874| 16444 9 8263 2754|119879119894119899V| 17111 9 8054 2685|119879119894119899V+119872119880119868119874| 16444 9 8263 2754

For asymmetric FSMs with only invertible transitions|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V+119872119880119868119874| are the same if the maximum-length IDOSs comply with the definition of FOTS Oth-erwise |119879119865119874119879119878+119872119880119868119874| tends to be longer because of moreappended UIO sequences as well as the possible extratransitions for concatenation 119879119894119899V is an Euler tour from arural symmetric augmentation of an asymmetric FSM withonly invertible transitions followed by a minimum-lengthUIO sequence of the initial state The number of appendedtransitions in the rural symmetric augmentation is a decidingfactor of |119879119894119899V|

For FSMs with noninvertible transitions |119879119865119874119879119878+119872119880119868119874| islonger than |119879119894119899V+119872119880119868119874| since every noninvertible transitionis verified individually by an appended UIO sequence |119879119894119899V|is the same as |119879119894119899V+119872119880119868119874| at best otherwise |119879119894119899V| tends to belonger than |119879119894119899V+119872119880119868119874|

As shown in Table 5 test sequences generated fromdifferent test methods conform to the above theoreticalanalysis In this section two-sample t-test is performed tocompare test methods in terms of the length of the associatedtest sequences Single sample K-S check in SPSS is used toverify the normality of the data studied since t-test is aparametric test and requires data to be normally distributedNormality results are reported whenever samples deviate

significantly from the normal distribution and the result ofsingle sample K-S check in Table 6 shows that all the datain this experiment follow normal distribution The pairedsamples statistics in Table 7 indicates that the average lengthof test sequences from the improved method is superior tothose of the other two methods

Vargha-Delaney effect size measure(11986012) is also calculatedto get more credible conclusions When comparing twomethods 11986012 measures the probability that one methodwould perform better than the other method A value of 05would mean that the two methods have equal probabilityof performing better than the other The Vargha-Delaneyeffect size measure (11986012) from comparing |119879119894119899V+119872119880119868119874| to|119879119865119874119879119878+119872119880119868119874| and |119879119894119899V| is shown in Table 8 The resultsshow that |119879119894119899V+119872119880119868119874| is statistically 605 and 543 ofthe time significantly shorter than |119879119865119874119879119878+119872119880119868119874| and |119879119894119899V|respectively

5 Related Work

There are many works on IoT testing Xiaoping Che et alpresented a logic-based approach to test the conformance andperformance of XMPP protocol which is gaining momentumin IoT through real execution traces and formally specified

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 11: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 11

Table 8 Statistical results from (11986012)

|119879119894119899V+119872119880119868119874| V119890119903119904119906119904 (11986012)

Ex1 |119879119865119874119879119878+119872119880119868119874| 0395Ex2 |119879119894119899V| 0457

properties [20] Dimitrios Serpanos et al introduced testingfor security for IoT systems and especially fuzz testing whichis a successful technique to identify vulnerabilities in systemsand network protocols [21] Martin Tappler et al presented amodel-based approach to test IoT communication via activeautomata learning [22] Combining Model-Based Testing(MBT) and a service-oriented solution Abbas Ahmad etal presented Model-Based Testing As A Service (MBTAAS)for testing data and IoT platforms [23] Hiun Kim et alintroduced IoT testing as a Service-IoT-TaaS which is com-posed of remote distributed interoperability testing scalableautomated conformance testing and semantics validationtesting components adequate for testing IoT devices [24]John Esquiagola et al used the current version of their IoTplatform to perform performance testing [25] Daniel Kuem-per et al described how concepts for semantically describedweb services can be transferred into the IoT domain [26]Philipp Rosenkranz et al propose a testing framework whichsupports continuous integration techniques and allows forthe integration of project contributors to volunteer hardwareand software resources to the test system [27] A connectiveand semantic similarity clustering algorithm (CSSCA) anda hierarchical combinatorial test model based on FSM areproposed by Kai Cui et al [28]

Test sequence generation and reduction has long beenan active research topic Porto et al used identification setswhich are subsets of a characterizing set to identify states andobtained reduced test sequences [29] Locating sequences areused to make sure that every element of a characterizing setis applied to the same state Jourdan et al generated shortertest sequences by means of reducing the number of locatingsequences [30] Baumgartner et al proposed a mixed integernonlinear programming (MINLP) model to formalize howthe total cost of testing depends on the sequence and theparameters of the elementary test steps [31] To provide anefficient formalization of the scheduling problem and avoiddifficulties due to the evaluation of an objective functionduring the relaxation of the integer variables the MINLPwas formulated as a process network synthesis problemHierons et al affirmed the importance of invertibility intest sequence reduction and considered three optimisationproblems associated with invertible sequences [32] Petrenkoet al addressed the problem of extending the checkingexperiment theory to cover a class of FSMs with symbolicextensions [33] They also reported the results that furtherlift the theory of checking experiments for Mealy machineswith symbolic inputs and symbolic outputs Hierons et aldescribed an efficient parallel algorithm that uses many-core GPUs for automatically deriving UIOs from Finite StateMachines [34]The proposed algorithm uses the global scopeof the GPUrsquos global memory through coalesced memory

access and minimizes the transfer between CPU and GPUmemory Song et al introduced a practical conformancetesting tool that generates high-coverage test input packetsusing a conformance test suite and symbolic execution Thisapproach can be viewed as the combination of conformancetesting and symbolic execution [35] Bokil et al presentedan automated black box test suite generation technique forreactive systems [36] The technique is based on dynamicmining of specifications in form of an FSM from initial runsThe set of test cases thus produced contain several redun-dant test cases many of which are eliminated by a simplegreedy test suite reduction algorithm to give the final testsuite

6 Conclusions

Taking Zigbee protocol as an example this paper introduceshow FSM-based conformance testing works in wireless pro-tocol conformance testing of IoT An improved method inwhich both overlapping by invertibility and multiple UIOsequences are considered is proposed to achieve test sequencereduction for wireless protocol conformance testing of IoTBased on invertibility transitions of all types of FSMs areverified with as few appended UIO sequences as possibleMultiple UIO sequences contribute to generate a shorter testsequence by means of reducing transitions for concatenationof test subsequences in the test sequence Moreover testsequences can be further reduced by removing the transitionsequence which follows the UIO sequence of the maximum-length IDOS that is verified last in the tourTheory and exper-iment indicate that the execution cost is reduced effectivelyby the improved method under the premise of not increasingthe generation cost The improved method is also applicableto traditional protocol conformance testing as well as reactivesystemsNumerous experimental data and practical examplesabout wireless protocols of IoT will be gathered in the futurework to analyze the effectiveness of the method

Data Availability

The data used to support the findings of this study areincluded within the article

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This work is supported by National Natural Science Founda-tion of China (Grant no 61572306 and Grant no 61502294)the IIOT Innovation andDevelopment Special Foundation ofShanghai (Grant no 2017-GYHLW-01037) and the CERNETInnovation Project (NGII20170513 and NGII20170206)

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 12: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

12 Security and Communication Networks

References

[1] L Qi X Zhang W Dou and Q Ni ldquoA distributed locality-sensitive hashing-based approach for cloud service recommen-dation from multi-source datardquo IEEE Journal on Selected Areasin Communications vol 35 no 11 pp 2616ndash2624 2017

[2] L Qi X Xu X Zhang et al ldquoStructural balance theory-based e-commerce recommendation over big rating datardquo IEEETransactions on Big Data vol 4 no 3 pp 301ndash312 2018

[3] K Zaniewski andW Pedrycz ldquoA hybrid optimization approachto conformance testing of finite automatardquo Applied Soft Com-puting vol 23 pp 91ndash103 2014

[4] S J S K H and P S ldquoEnhancing conformance testing usingsymbolic execution for network protocolsrdquo IEEE Transactionson Reliability vol 64 no 3 pp 1024ndash1037 2015

[5] P Liu H-K Miao H-W Zeng and Y Liu ldquoFSM-based testingTheory method and evaluationrdquo Jisuanji XuebaoChinese Jour-nal of Computers vol 34 no 6 pp 965ndash984 2011

[6] R Dorofeeva K El-Fakih S Maag A R Cavalli and NYevtushenko ldquoExperimental evaluation of FSM-based testingmethodsrdquo in Proceedings of the 3rd IEEE International Confer-ence on Software Engineering and Formal Methods (SEFM rsquo05)pp 23ndash32 IEEE Koblenz Germany September 2005

[7] X Zhang M Yang J Zhang H Shi and W Zhang ldquoA studyon the extended unique inputoutput sequencerdquo InformationSciences vol 203 pp 44ndash58 2012

[8] I Ahmad F M Ali and A S Das ldquoSynthesis of finitestate machines for improved state verificationrdquo Computers ampElectrical Engineering vol 32 no 5 pp 349ndash363 2006

[9] J Shujuan J Xiaolin W Xingya L Haiyang Z Yamei andL Yingqi ldquoMeasuring the importance of classes using uiosequencerdquo Chinese Journal of Electronics vol 43 no 10 pp2062ndash2068 2015 (Chinese)

[10] E F D Souza V A D Santiago Junior and N L VijaykumarldquoH-Switch Cover a new test criterion to generate test case fromfinite state machinesrdquo Software Quality Journal vol 25 no 2pp 373ndash405 2017

[11] B Yang and H Ural ldquoProtocol conformance test generationusing multiple uio sequences with overlappingrdquo ACM SIG-COMM Computer Communication Review vol 20 no 4 pp118ndash125 1990

[12] R M Hierons ldquoExtending test sequence overlap by invertibil-ityrdquoThe Computer Journal vol 39 no 4 pp 325ndash330 1996

[13] J Bang-Jensen and G Z Gutin Digraphs Theory Algorithmsand Applications Springer New York NY USA 2002

[14] S Mouchawrab L C Briand Y Labiche and M Di PentaldquoAssessing comparing and combining state machine-basedtesting and structural testing a series of experimentsrdquo IEEETransactions on Software Engineering vol 37 no 2 pp 161ndash1872011

[15] K El-Fakih A Simao N Jadoon and J C Maldonado ldquoAnassessment of extended finite state machine test selectioncriteriardquoThe Journal of Systems and Software vol 123 pp 106ndash118 2017

[16] A T Endo and A Simao ldquoEvaluating test suite characteristicscost and effectiveness of FSM-based testingmethodsrdquo Informa-tion and Software Technology vol 55 no 6 pp 1045ndash1062 2013

[17] J F Cutigi A Simao and S R Souza ldquoReducing fsm-based testsuites with guaranteed fault coveragerdquo The Computer Journalvol 59 no 8 pp 1129ndash1143 2016

[18] W-W Lin andH-W Zeng ldquoA chain algorithm for conformancetesting based on uio sequencesrdquo in Proceedings of the Interna-tional Conference on Software Engineering Artificial IntelligenceNetworking and ParallelDistributed Computing pp 1ndash6 IEEE2015

[19] P Liu H-K Miao H-W Zeng and J Mei ldquoDFSM-basedminimum test cost transition coverage criterionrdquoRuan JianXueBaoJournal of Software vol 22 no 7 pp 1457ndash1474 2011

[20] X-P Che and SMaag ldquoA passive testing approach for protocolsin internet of thingsrdquo in Proceedings of the 2013 IEEE Inter-national Conference on Green Computing and Communicationsand IEEE Internet of Things and IEEE Cyber Physical and SocialComputing pp 678ndash684 IEEE 2013

[21] D Serpanos and M Wolf ldquoSecurity testing iot systemsrdquoInternet-of-Things (IoT) Systems pp 77ndash89 2018

[22] M Tappler B K Aichernig and R Bloem ldquoModel-BasedTesting IoT Communication via Active Automata Learningrdquoin Proceedings of the 10th IEEE International Conference onSoftware Testing Verification and Validation ICST 2017 pp276ndash287 IEEE March 2017

[23] A Ahmad F Bouquet E Fourneret F L Gall1 and BLegeard ldquoModel-based testing as a service for iot platformsrdquoin Proceedings of the International Symposium on LeveragingApplications of Formal Methods pp 727ndash742 Springer 2016

[24] H Kim A Ahmad J Hwang et al ldquoIoT-TaaS Towards aProspective IoT Testing Frameworkrdquo IEEE Access vol 6 pp15480ndash15493 2018

[25] J Esquiagola L Costa P Calcina G Fedrecheski and MZuffo ldquoPerformance testing of an internet of things platformrdquoin Proceedings of the 2nd International Conference on Internetof Things Big Data and Security IoTBDS 2017 pp 309ndash314Portugal April 2017

[26] K Daniel E Reetz and R Tonjes ldquoTest derivation for seman-tically described iot servicesrdquo in Proceedings of the 2013 FutureNetwork amp Mobile Summit pp 1ndash10 2013

[27] P Rosenkranz M Wahlisch E Baccelli and L OrtmannldquoA Distributed Test System Architecture for Open-source IoTSoftwarerdquo in Proceedings of the 2015Workshop on IoT challengesin Mobile and Industrial Systems pp 43ndash48 ACM FlorenceItaly May 2015

[28] K Cui K-j Zhou T Qiu M-c Li and L-m Yan ldquoA hierar-chical combinatorial testing method for smart phone softwarein wearable iot systemsrdquo Computers and Electrical Engineeringvol 61 pp 250ndash265 2017

[29] F R Porto A T Endo and A Simao ldquoGeneration of checkingsequences using identification setsrdquo in Proceedings of the Inter-national Conference on Formal Engineering Methods pp 115ndash130 Springer 2013

[30] G-V Jourdan H Ural and H Yenigun ldquoReduced checkingsequences using unreliable resetrdquo Information Processing Let-ters vol 115 no 5 pp 532ndash535 2015

[31] J Baumgartner Z Sle B Bertk and J Abonyi ldquoTest-sequenceoptimisation by survival analysisrdquo Central European Journal ofOperations Research pp 1ndash19 2018

[32] R M Hierons M R Mousavi M K Thomsen and U CTrker ldquoHardness of deriving invertible sequences from finitestate machinesrdquo in Proceedings of the International Conferenceon Current Trends inTheory and Practice of Informatics pp 147ndash160 Springer 2017

[33] A Petrenko ldquoToward testing from finite state machines withsymbolic inputs and outputsrdquo Software and Systems Modelingpp 1ndash11 2017

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 13: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

Security and Communication Networks 13

[34] RMHierons andU C Turker ldquoParallel Algorithms for TestingFinite StateMachinesGeneratingUIO Sequencesrdquo IEEE Trans-actions on Software Engineering vol 42 no 11 pp 1077ndash10912016

[35] J Song H Kim and S Park ldquoEnhancing Conformance TestingUsing Symbolic Execution for Network Protocolsrdquo IEEE Trans-actions on Reliability vol 64 no 3 pp 1024ndash1037 2015

[36] P Bokil P Krishnan and R Venkatesh ldquoAchieving effectivetest suites for reactive systems using specification miningand test suite reduction techniquesrdquo ACM SIGSOFT SoftwareEngineering Notes vol 40 no 1 pp 1ndash8 2015

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 14: Test Sequence Reduction of Wireless Protocol Conformance ...downloads.hindawi.com/journals/scn/2018/3723691.pdf · in the test sequence, leading to a reduced test sequence. In other

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom