Testing Generation at UPenn

Model-Based Test Generation

Temp. Prop. Translator

ControllerModel Checker----------------------

Witness generator

1 Æ ……Æ ni

i ² i

Concretizer

TS={1,……, n}

Specification Model

Implementation

Tester demand/Specification

Test Suite

Test Setting = temporal properties +

specification + implementation

Coverage Criteria

Testing Generation at UPenn

Model-Based Test Generation

Overview: Use witness-ready model checker to automate the

test generation. Flexible: properties can mimic the traditional coverage

criteria, or can be directly specified by tester. Efficient: Efficiency inherent from checker’s search

engine. Effective: fine-tune test by specifying extra constrains

f=G ( (begin ) F end) Æ (setValue ) F move)) Add: f1=F (setValue): Test fÆ f1

Current research Testing discrete systems.

Mimicing traditional coverage criteria Is in general LTL property “testable”?

Testing hybrid systems. Randomized simulation approach Reachability checker-assisted approach.

Testing Generation at UPenn

Testing discrete systemsGiven: Test setting = LTL/9 LTL + the

specification+ Blackbox implementation.Problem: Currently testing properties is limited

to 9LTL with eventuality only.Question: is there a test for “F( G( a ! Xb))”. Require to test all the possible executions May require a test with infinite length.

Testing Generation at UPenn

Property-Coverage Testing Synthesizing test suites for 9LTL property.

1. Is the property “E GF a” testable?1. No finite trace can be attest to this property.

2. If the number of states in blackbox is bounded by n,

1. A trace for 9 LTL + the specification is rational: ().

2. A infinite trace () can be cut to ()n

3. Buchi tree automaton-based model checker can be used to generate rational traces.

Synthesizing test suites for LTL property.1. LTL can be translated to a set of interesting 9 LTL

properties.1. E( GF( a) Æ F(G(a ) X b)) is an interesting property

for F(G(a ) X b)) 2. Each interesting 9 LTL property focuses on

testing a particular portion of LTL formula.

Testing Generation at UPenn

Testing Hybrid System: Phase I

Randomized test generator=Randomized Simulator+ Coverage Checker. 1. Local ramdomization, gobal strategy.

1. Stay or jump2. Where to jump3. How long to stay

2. Gobal ramdomization1. Aborting/Continuing on current trace.

Mode Adf/dt=1

a: True:f=0b: 1· f<3:m=1

c: 2 · f<4:m=2

Mode Bdw/dt=1

Mode C

d: 2 · f<5:m=4

Testing Generation at UPenn

Testing Hybrid System: Phase I

1. Heuristic search1. Uncovered neighbor first2. Syntax-based distance matrix (Shortest distance to uncovered state/location)3. Open question: Make local decision based global information/history.

1. deciding the weight for outgoing transitions based on the history (What should we learn from a failed search).

2. Deciding the duration to stay in a mode.2. Current status: a working version of randomized test generation is written on CHARON simulator.

Testing Generation at UPenn

Testing Hybrid System: Phase II

SystemModeling

CHARON(Model) Flatten hybrid model

Concretizer

Implementation

Test Suite

Set of predicates

Coverage criteria

Bad set

Reachability Checker

Yes w/ TraceSimulation/refinment

NO w/ more predicates

YES

No

Testing Generation at UPenn

Intelligent simulatorIntelligent simulator=simulator+ property checker

(monitor)1. Verification as the byproduct of simulation

1. LTL Property encoded as the monitor1. MEDL: A subset of LTL, has been applied to Java running-

time monitoring.

2. Monitor advances when the simulation proceeds.3. Open problem: LTL with eventuality only is easy, but

how about other formula requires circularity reasoning. 1. Need to remember the states traversed to sense the loop.

1. Difficult because the domain of continuous variables are dense.

2. The search is tailored by the property.1. A transition “measure” has the priority higher than

others if the property is G(measure => X (home)). 2. Most interesting simulation trace: Covering as many

parts of property as possible using less steps.