automating software testing at siemens mppeople.cs.aau.dk/~bnielsen/mbtv07/material/mtv-smp.pdf ·...

Automating Software Testing at Siemens MP

Brian Nielsen, [email protected] of Computer Science,

Aalborg University, Denmark

Agenda

• RRLP case• (ASN.1)• MSC• TTCN• (Translation of MSC to TTCN)

RRLP case study

Case Study• USA FCC Wireless E911 Rules• Enabling technology for future

Location Based Services• RRLP = Radio Resources Location Services • Protocol for communicating Location

Information and/or Measurements to network from MS.

• Supports • MS Based and MS Assisted types, and• GPS and EOTD measurement methods

E-OTD

d

d

1

2

BaseStation

Base Station

Measurementerror margin

Base Station

LCS Architecture

MS MSC/VLR

HLR

SMLC

LeLg

Lg

Lh

Other PLMN

Um Ls

BSC AAbis

Lb

SMLCLp

Abis

gsmSCF

Lc

CBCLMUType A

CBC-SMLC

CBC-BSC

BTS(LMU

Type B)

ExternalLCS client

GatewayMLC

GatewayMLCLMU

Type B

RRLP

= highest layer where segmentation of upper layers is supported

Target MS BSC MSC SMLC

L1

L2( LAPDm)

RRLP(04.31)

RRLP(04.31)

SCCP

MTP

SCCP

MTP

SCCP

MTP

SCCP

MTPL1

L2(LAPD)

Relay

Um A Ls

BSSLAP(08.71)

BSSLAP

Relay

RR(04.08) RR

BSSAP BSSAP BSSAP-LE

BSSAP-LE(09.31)

+RRAP = Radio Ressources Application Protocol

RRLP• In principle a simple request/reply protocol!• Specification distributed across several ETSI

GSM standards• Vague, ambiguous, implicit • Detailed domain knowledge and

understanding of surrounding protocols required

• Long, complex parameter lists • Message syntax takes 12 A4 pages of ASN.1

Type Declarations(+syntax errors)!!

Baroque!!

• baroque adj. [common] Feature-encrusted; complex; gaudy; verging on excessive. Said of hardware or (esp.) software designs, this has many of the connotations of elephantine or monstrosity but is less extreme and not pejorative in itself.

http://info.astrian.net/jargon/terms/b/baroque.html

Ambigious1. When MS receives a RRLP message, that contains IEs or

elements of components that are syntactically incorrect, theMS sends a Protocol Error component with indication”incorrect data”

2. ”When after the reception of a Masurement Position Request component, but before responding with a MeasurementPosition Response or Protocol Error, the MS receives a new RRLP message with the Measure Position Request component, it acts as follows

A. If the old and new Measurement Request Components have thesame reference number, the latter is ignored.

B. If the old and new Measure Position Request components havedifferent reference numbers, the MS aborts the formercomponent, and starts to act according to the latter componentand sends a response to that. ”

• [GSM 4.31 v 8.6.0]

Example 1MSSS

RRLP_Position_Request

(1, validRequest_Component)


(1, InvalidRequest_component)

RRLP_Position_Response

(1,Valid_measurement)

RRLP_Protocol_Error

(1,IncorrectData)

Example 1MSSS







RRLP_Protocol_Error

(1,IncorrectData)

•Protocol Error due rule1!

Example 1MSSS







RRLP_Protocol_Error

(1,IncorrectData)

•Protocol Error due rule1!•Ignored due to rule 2.A,hence Position Response!

Example 1MSSS







RRLP_Protocol_Error

(1,IncorrectData)

•Protocol Error due rule1!•Ignored due to rule 2.A,hence Position Response!•Both protocol Error and Response,(robustness)

Example 2

•Protocol Error due rule1 and 2b

MSSS





RRLP_Protocol_Error

(2,IncorrectData)



•Both protocol Error and Response(robustness)

Test Equipment• 2*CRTG (4 channels) 2 * 200 k€

Test Equipment• Complete Type Approval Test System (3 M€)

Simulated Net Configuration

A

B C

YX

÷÷Y

C

÷X

÷Z

B

A

Visi-ble?

Neigh-bor?BTS

Z

Aspects to test• RRLP behavior FSM • Message en/decoder (PER)• Interaction with underlying protocols (RRAP

and RR) • Integration of RRLP and EOTD computation

component• Precision

• Precision of measurement results• Integration of EOTD-computation with low level

(DSP) SW• Correct calculations of EOTD component

Test situations• RRLP measurement during

• Emergency Call and Normal call• RR channel Established, but no call• SIM card present/absent• SMS received during call• Before, after, and during handover• Call waiting / holding• Different combinations of number and visible BTSs• ...

Rohde & Schwartz Testcase...

/*---------------------------------------------*//* Wait for CONNECT and send CONNECT ACK *//*---------------------------------------------*/

if (!CallAccepted(&TchCarrier,(tpByte)&L3Message,

MessagePool[CONNECT_ACK],&TI,&FrameTxd,&FrameRxd))

{ exit(FAILURE); }

/*---------------------------------------------*//* Wait for Instruction to Clear Call *//*---------------------------------------------*/

if (!UserAction("\nPress a key to Terminate Call\n”,KEYPRESS_REQUIRED))

{ exit(FAILURE); }

/*---------------------------------------------*//* Perform Call Clearing initiated by Network *//*---------------------------------------------*/

if (!CallClearing(&TchCarrier,MessagePool[DISCONNECT],

(tpByte)&L3Message,MessagePool[RELEASE_COMPLETE],&TI))

{ exit(FAILURE); }

/*3 0 3 f

Message file template - CONNECT_ACK*/#define CONNECT_ACK 8

/*6 0 3 25 2 e0 90

Message file template - DISCONNECT*/#define DISCONNECT 9

/*3 0 3 2a

Message file template- RELEASE_COMPLETE*/#define RELEASE_COMPLETE 10

Telecom Languages• ASN.1

• Abstract Syntax Notation 1• Defines messages syntax and contents

• TTCN• Tree and Tabular Combined Notation • Test case programming language

• MSC • Message Sequence Charts• Describes interactions between entities

• SDL• Specification and design language• Models system structure and behavior

ASN.1

MSC/HMSC

TTCNSDL

ASN.1• Abstract Syntax Notation One• ITU-T X.680 and X.690 series

• ISO/IEC 8824 and 8825• For specification of data (e.g. Messages)

• syntax of messages and other data structures• encoding of messages and other data structures

• Primitive types• INTEGER, BOOLEAN, IA5String, BIT STRING, • etc.,

• Type constructors• structures (SEQUENCE), • lists (SEQUENCE OF), • choice between types (CHOICE), • etc.

▼

ASN.1• Fully integrated in the formal languages SDL and

TTCN• Several associated standardized encodings, such as:

• efficient (binary) encoding: Packed Encoding Rules (PER)• canonical encoding for digital signatures:

Distinguished Encoding Rules (DER)• XML encoding rules (XER)

• Offers extensibility and interworking between previously deployed systems and newer, updated versions designed years apart

• ASN.1 website: http://asn1.elibel.tm.fr

ASN.1Types, Values and BER

Abstract: message as it appears in an ETSI standard

(X.680)

Concrete: message as it is transferred over a real network

(X.690)

MSG::= SEQUENCE{ item1 INTEGER,

item2 STRING }

Type

MSG m1 { item1 123,item2 "XYZ" }

Value 1MSG m2 { item1 789,

item2 "ABC" }

Value 2 MSG m3 { item1 123,item2 “ABC”

}

Value 3

11101001010011010010111…

Bitstream

▼

ASN.1Types, Values and BER

Abstract: message as it appears in an ETSI standard

(X.680)

Concrete: message as it is transferred over a real network

(X.690)

Encoding Rules

MSG::= SEQUENCE{ item1 INTEGER,

item2 STRING }

Type

MSG m1 { item1 123,item2 "XYZ" }

Value 1MSG m2 { item1 789,

item2 "ABC" }

Value 2 MSG m3 { item1 123,item2 “ABC”

}

Value 3

11101001010011010010111…

Bitstream

▼

RRLPPDU ::= SEQUENCE {referenceNumber INTEGER (0..7),component RRLPComponent

}

RRLPComponent ::= CHOICE {msrPositionReq MsrPosition_Req,msrPositionRsp MsrPosition_Rsp,assistanceData AssistanceData,assistanceDataAck NULL,protocolError ProtocolError,... -- extension marker! }

MsrPosition_Req ::= SEQUENCE {positionInstruct PositionInstruct,referenceAssistData ReferenceAssistData OPTIONAL,msrAssistData MstAssistData OPTIONAL,systemInfoAssistData SystemInfoAssistData OPTIONAL,gps_assistData GPS_AssistData OPTIONAL,extensionContainer ExtensionContainer OPTIONAL,…,rel98MsrPositionReq_Extension Rel98MsrPositionReq_Ext OPTIONAL

}

ASN.1 RRLP Types GSM 4.31

And so forth for 12 A4 pages

GSM 4.31 (contd.)PositionInstruct ::= SEQUENCE {

-- Method type methodType MethodType,positionMethod PositionMethod,measureResponseTime MeasureResponseTime,useMultipleSets UseMultipleSets,environmentCharacter EnvironmentCharacter OPTIONAL

}MethodType ::= CHOICE {

msAssisted AccuracyOpt,msBased Accuracy,’msBasedPref Accuracy,msAssistedPref Accuracy

}AccuracyOpt ::= SEQUENCE { accuracy Accuracy OPTIONAL }Accuracy ::= INTEGER (0..127) PositionMethod ::= ENUMERATED {

eotd (0),gps (1),gpsOrEOTD (2)

}

ASN.1 Value Notation (RRLP)aRRLPMessage RRLPPDU ::= {

referenceNumber: 1 ,rr_component msrPositionReq: {

positionInstruct: { methodType msAssisted_brian: { } ,positionMethod eotd_carsten,measureResponseTime 7 ,useMultipleSets oneSet }

referenceAssistData {bcchCarrier 50 ,bsic 1 ,timeSlotScheme equalLength } ,

msrAssistData {msrAssistList { {

bcchCarrier 70 ,bsic 3 ,multiFrameOffset 50 ,timeSlotScheme equalLength,roughRTD 1249 } ,

{ bcchCarrier 60 , bsic 2 , multiFrameOffset 13 , timeSlotScheme variousLength, roughRTD 13 }

} } } }

PER Example (1)(octet 0)

001 Sequence numberRRLP component

0 Extension bit: No extension present000 Choice number 0: position request

Position Request 0 Extension bit: No extension present(octet 1)

11000 Option bit map: referenceAssist present,msr assist presentPosition Instruction

0 option bit map : env character not present 00 Choice methodtype ms assisted. (octet 2)

0 Option bit map: Accuracy not present00 Position method (eotd=0)111 Measure response time = 7 1 Use multiple sets = one set.

Reference AssistData 0 Option bit map: BTS position Not Present(octet 3)

00001100,10 BCCH 50 000001 BSIC 1

PER Example (2)

=‘001000001100000000011110000011001000000100001000010001100000111100100100111000010000011110000001000110110000000110100000'B

(octet 5)0 Time slot Scheme

MSR Assist Data0001 LENGTH = 2 (2-1: NB Effective length of 1..15 is Len-1)

MSR Assist BTS 10 Option Bit Map:calc Assistance Data not present00,01000110 BCCH 70

(octet 7)000011 BSIC 311,0010 Multi Frame Offset 50 0 time Slot Scheme (equal Length) 100,11100001 Rough RTD 1249(octet 10)

MSR ASSIST BTS 20 Option Bit map: calc Assistance Data Not Present0000111,100 BCCH 60 00001,0 BSIC 20 001101 multi Frame Offset 131 Time Slot Scheme Various Length (octet 13)

00000001,101 Rough RTD00000 padding for octet alignment

MSC

Message Sequence Charts

MSC• Message Sequence Charts• ITU-T Z.120• For specification of message flows

• Component interactions• Interface boundaries• sequential ordering of messages, timers

etc.• Releases 1990, 1994, 1996 and 2000

▼

Application of MSCs

• Requirements capturing• Use cases• Behavior Modelling• Trace visualization• Property language• Test case specifications• Test purpose specification

MSC behavior• Instances, events, and actions• Instance creation, termination• Timers• Conditions• Co-regions• Causal ordering• Inline expressions• Lost and found messages

MSC Timers and Conditions

Phone PhoneCO

T1(0.5s) OffHook

DialTone

Dial

(96358883)

ringing

connected

Party_responds

T1(60s)

MSC Dial

settingconditions

cancel timer

timeout

MSC dynamics• No notion of channels or buffering• “every message has an unbounded

FIFO queue”• Logical time

• Events on an instance are totally ordered

• Send(m) before receive (m)

i1:send(m1) before i2:recv(m1)i1:send(m1) before i1:recv(m3)i2:recv(m1) before i2:send(m2)i2:send(m2) before i2:anActioni2:send(m2) before i3:recv(m2)i2:anAction before i2:send(m3)i2:send(m3) before i1:recv(m3)i1:recv(m3) concurrent i3:recv(m2)

i1

MSC basic

i2 i3

m3

m1m2

anAction

i1

MSC overtake

i2

m1

m2

MSC Ordering Concepts

Phone PhoneCO

WHEN connected

Talk

MSC Talk

(bla. bla bla)

Talk(yak, yak, yak)

Phone PhoneCO

WHEN connected

Talk

MSC Talk

(bla. bla bla)

Talk(yak, yak, yak)

ALT

(bla. bla bla)

Talk

(yak, yak, yak)

Talk

+specification of causal ordering of events / actions

co-region = ”any order”

choiceguarding conditions

Inline Expressions

• Choice• Parallel• Loop• Optional• Exceptional

ALT

PAR

OPT

EXC

LOOP<L,U>

Structural Concepts

• MSC references• Gates• High Level MSCs• Instance Decomposition

MSC references

Phone PhoneCO

MSC Phonecall

Dial

Talk_A2B par Talk B2A

Reference expression operators•par•alt•seq•loop•exc•opt•empty

Gates• ”Export / Import” mechanism for messages

Phone_A PhoneCO_A

MSC Phone_System

PhoneCO_B Phone_B

Relay_call

PhoneCO_A

MSC Relay_call

PhoneCO_B

Dial

(96358883)

Ring

Dial

(96358883)

Ring

Dial

(8883)g1 g2

g1

g2

actualinput gate

actualoutput gate

input gatedefinition

output gatedefinition

High-Level MSCs• Graphical overview of the

relation between MSCs• Focus on control structure• Constructs for

• Looping• Sequential composition• Alternative (sequential

composition)• Parallel composition• Conditions

connectionRequest

fail established

DataTransmission

MSC Transmit

MSC 2000

• Improved Data Support• Interface for data type definition languages• Variable declarations• Parameterized MSCs

• Synchronous Operations• MSC 2000 to be adopted as UML SCs?

Behavior TreesProcess 1

receive(request1)//process reqsend(response1)

Process 2


Implementation

p1

p2

Tester

send (request1) to p1send (request2) to p2EXPECT(response1) ??EXPECT(response2) ??



Process 2


Implementation

p1

p2

Tester


Message Ordering is NON-Deterministic ~ Unpredictable



Process 2


Implementation

p1

p2

Tester



Test Cases must branch = Trees



Process 2


Implementation

p1

p2

Tester



Test Cases must branch = Trees

B

C

D E

Behavior Tree

A

request2

response1 response2

request1

Tree and Tabular Combined Notation

• For specification of tests (of systems)• test architectures• communication to the object being tested • test behaviour based on trees

• An ISO/ITU-T standard (adopted by ETSI)• ISO/IEC 9646-3• X.290 Series

• Version 1, 2 and now 3• Two formats

• Graphical (GR)• Machine Processable (MP)

Main Components of TTCN• Test Suite Structure

• Test Suite, Test Groups, Test Case, Test Steps• Data declarations

• Points of Control and Observation (PCO’s),• Abstract Service Primitives (ASP’s)• PDU Data Types (TTCN data types / ASN.1)

• Constraints• Actual instances of PDUs to be sent• Patterns of PDUs expected to be received

• Dynamic behaviour trees (Test Cases)• Send, Receive, Timers, Expressions, Qualifiers• Test Steps

TTCN Tests Reactive Systems• Message Based Communication• Asynchronous• PCO = Point of control and

observation ≈ bi-directional message Q

• PDU = Protocol Data Unit ≈message

• ASP = Abstract Service Primitive ≈ “procedure invoked by message”

• L!A ≈ send A via pco L• L?X ≈ receive A via pco L

PCO L

IUT

TTCNTester

L!A L?X

Tree and Tabular

E7I8J9

F3G4

H5D6

B10

C2A1

verdictconstraintBehaviorLabelNr

A B

C D E

F G

H

I J

Behavior Tree

Alter-natives

Sequence

•Same indentation level = Alternatives•Increasing indentation level = Sequence

Example of a Test Case

Test Case Dynamic BehaviourTest Case Name : TC1Group : A_SUITE/GROUP1/GROUP2Purpose : An exampleDefaults : DEF1Comments :Nr L Behaviour Description Constraint Ref V Comments

12345678910

[X=1]L! PDU_1 START T1

L? PDU_2 CANCEL T1?OTHERWISE?TIMEOUT T1

[X=2]L! PDU_3 (A:=A+1)

L? PDU_4+ A_Test_Step (3)

?OTHERWISE

P1P2

P3P4

PFI

P

F

TTCN statements• Send Statement

• L!N_DATArequest [B=1] (X:=3) START T1• If b=1 then X:=3 then l!N_DataRequest followed by START T1

• Receive Statement• L?N_DATArequest [B=1] (X:=3) CANCEL T1• If a matching N_DATArequest message is at the head of L and if

[B=1] then (X:=3)followed by CANCEL T1

• Timeout Statement• T1?Timeout [B=1] (X=readTimer(T)) START T2

• Otherwise Statement• L?OTHERWISE • Catch all (if no listed matching alternative exist)

• Goto statement• Goto <label>

TTCN Constraints• PDU / ASP type definition

• Example constraints

PDU::= SEQUENCE {sequenceNo INTEGER (0..20),Item2 INTEGER OPTIONAL}

exampleConstraint PDU {sequenceNo 5,Item2 7}

invalidSeq PDU {sequenceNo 0,Item2 * -- anyOrOmit}

validSeq PDU {sequenceNo (1..20), --rangeItem2 ? -- any Value}

response5 PDU {sequenceNo 5, Item2 expectedItem5 --predefined constraint }

TTCN matching mechanisms

413(3,5,7,13,17)INTEGERValue List

87, -7 IF_PRESENTINTEGER OPTIONAL

IF_Present

{2}, {2,3,4,5}{3,2,1}, {1}{PERMUTATION(1,*) }

SEQUENCE OFINTEGER

PERMUTATION

{1,4,2}{3,2,1}{PERMUTATION(1,2,3) }

SEQUENCE OFINTEGER

PERMUTATION

0, 11’01’B

5’111’B

??

INTEGER (1..10)BITSTRING[3]

AnyValue

1,205, ’110’B(2..6)INTEGER (1..10)Value Range

199INTEGERSpecific Value

NonMatchingValue

MatchingValue

ConstraintValue

TypeMatchingMechanism

GSM Layer 1 & 2

MSIUT

SS

RRLP Test Architecture

PCO

dial 112wait for ”mobile in service”

MTC

PCO

RRLP_PositionRequestRRLP_PositionResponse

...

SUT

RRLP Test Case

RRLP non-determinism• MS receives second

while performing measurement

• MS receives second request after completing first request

Mobile Station

RRLP EOTD

RRLP test case 2

Position Response Constraintc_Rsp_1__ValidEOTDMeasurementResult_(refNo: ReferenceNumber){ rRLPPDU1 {

referenceNumber 1 , rr_component msrPositionRsp:{referenceIdentity ref_identity( lac_BTS_A , cellID_BTS_A),

otd_MeasureInfo validotd_ref_A}

}}

RRLP TTCN Constraintvalidotd_ref_A OTD_MeasureInfo{otdMsrFirstSets {

refFrameNumber ? , -- AnyValuereferenceTimeSlot ? ,toaMeasurementsOfRef {

refQuality ? ,numOfMeasurements ?

} ,stdResolution ?,taCorrection *, -- AnyOrOMITotd_FirstSetMsrs ({ --value list

otd_msr( bcch_BTS_C , bsic_BTS_C),otd_msr( bcch_BTS_B , bsic_BTS_B),},{otd_msr( bcch_BTS_B , bsic_BTS_B),otd_msr( bcch_BTS_C , bsic_BTS_C),})

}-- OMITted fields are not allowed

}

TTCN-3• Synchronous operations• Notations

• MSC 2000 is a graphical notation• Tabular format• Textual format :More like a programming language

(Concepts for PCO, PTC, Verdicts etc)• Support for other IDLs than ASN.1• Improved matching, type and value

parameterization• Ability to specify encoding information

http://www.itu.int/ITU-T/studygroups/com17/ttcn.htmlhttp://www.etsi.org/ptcc/home.htm (ETSI Protocol and TestingCompetence Center)

TTCN-3• ETSI conformance test specifications are in TTCN. • TTCN-3 developed by ETSI Technical Committee MTS (Methods for

Testing and Specification). • Application Domain

• protocol testing including mobile and Internet protocols• supplementary service testing,• module testing,• testing of CORBA based platforms,• testing of APIs etc.

• TTCN-3 is no longer restricted to conformance testing. • interoperability,• robustness (torture),• regression, • system and integration testing.

function PO49901(integer FL) runs on MyPTC {L0.send(A_RL3(FL,CREF1,16));TAC.start;alt {

[] L0.receive(A_RC1((FL+1) mod 2)) {TAC.cancel

}[] TAC.timeout {

verdict.set(inconclusive)}

[] any.receive {verdict.set(inconclusive)

}}END_PTC1() // Function call

}

TTCN-3 Test case

Synthesizing Test cases

Translation of MSCs to TTCNStructural model of test interface(SDL)

Structural model of test interface(SDL)

AutoLinkTranslator

AutoLinkTranslator

Test –SuiteTTCN

Test –SuiteTTCN

ASN.1•Types•Values

ASN.1•Types•Values

MSC’s are thetest-casesMSC’s are thetest-cases

TTCNMasterTTCNMaster

Translation rules•Maps abstract parameter lists to ttcn constraint names•Parameterizes constraints•Places test cases in test groups

TTCN “Master” file •Predefined constraints / values•TTCN test cases

TTCNEditorMerge

TTCNEditorMerge

RRLP Example

RRLP example 2

Modeling test steps

RRLP non-determinism• MS receives second

while performing measurement

• MS receives second request after completing first request

Mobile Station

RRLP EOTD

automating software testing at siemens mppeople.cs.aau.dk/~bnielsen/mbtv07/material/mtv-smp.pdf ·...

Documents