Quantitative Trace Analysis using

Extended Timing Diagrams

Andreas Richter and Klaus Kabitzsch

Dresden University of Technology, Institute of Applied Computer Science,Chair of Technical Information Systems, D-01062 Dresden, Germany

RV12 - September 26, 2012

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

MotivationRequirements

MotivationEmbedded Systems in the Automotive Domain

• Application of networked embedded controllers haspermanently increased over the last decades

• Cars are complex reactive systems

• Up to 100 electronic control units (ECU)• Over 1000 (distributed) functions• More than 100 MB control application code

• ECU strongly interconnected

• CAN, FlexRay, LIN, MOST• Ethernet, Powerline, WLAN

• Ever increasing complexity

• Hybrid technologies• Driver assistance, Car-IT• Car2Car Communication

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 2/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

MotivationRequirements

Quality AssuranceTesting vs. Trace Analysis

• Veri�cation and quality assurance are key topics in these domains

• Development and testing processes with strong tool support

• Many problems not detectable or avoidable by traditional testingand diagnosis

• Complex driver and environment behaviour• Reactivity of systems• Real-time aspects• ↪→ Sporadic and transient errors• ↪→ Non-reproducible faults

B Increase degree of validation through runtime monitoring andsubsequent trace analysis

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 3/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

MotivationRequirements

Quality AssuranceTesting vs. Trace Analysis

• Veri�cation and quality assurance are key topics in these domains

• Development and testing processes with strong tool support

• Many problems not detectable or avoidable by traditional testingand diagnosis

• Complex driver and environment behaviour• Reactivity of systems• Real-time aspects• ↪→ Sporadic and transient errors• ↪→ Non-reproducible faults

B Increase degree of validation through runtime monitoring andsubsequent trace analysis

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 3/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

MotivationRequirements

Runtime Veri�cation for Industrial ApplicationRequirements

Veri�cation Methods

• Typical questions

• How often did a pattern occur?• When and to which extent was a timing constraint violated?• Is an event drifting towards one endpoint of a given timing interval?

• Quantitative trace analysis can answer those!

• Operates on concrete system executions• Access to concrete system state values and timings

Speci�cation Languages

• Su�cient expression power (↗ quantitative properties)

• Understandable and usable for practitioners (visual formalisms)

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 4/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

MotivationRequirements

Runtime Veri�cation for Industrial ApplicationRequirements

Veri�cation Methods

• Typical questions

• How often did a pattern occur?• When and to which extent was a timing constraint violated?• Is an event drifting towards one endpoint of a given timing interval?

• Quantitative trace analysis can answer those!

• Operates on concrete system executions• Access to concrete system state values and timings

Speci�cation Languages

• Su�cient expression power (↗ quantitative properties)

• Understandable and usable for practitioners (visual formalisms)

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 4/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Visual Speci�cationTiming Diagrams (TD)

• Established speci�cation language in the engineering domains

• Focus on the states of signals, state changes and correspondingtiming relations

• Speci�cation of I/O behaviour of networked systems

• TD resemble the signal display of oscilloscopes

• Former applications

• Interaction modelling during software design (UML2)• Speci�cation language for model checking [Fis99]• Veri�cation of hardware designs [DJS95]

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 5/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Visual Speci�cationTiming Diagrams (TD)

• Established speci�cation language in the engineering domains

• Focus on the states of signals, state changes and correspondingtiming relations

• Speci�cation of I/O behaviour of networked systems

• TD resemble the signal display of oscilloscopes

• Former applications

• Interaction modelling during software design (UML2)• Speci�cation language for model checking [Fis99]• Veri�cation of hardware designs [DJS95]

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 5/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Timing DiagramsSyntax Overview

• One or more signals along the vertical axis

• Timeline running in positive horizontal direction

• Signals hold waveforms, formed by a sequences of edges

• State expressions constrain expected signal values

• Only partial order of state value changes

• Relationships (arrows) with interval annotations [min,max] specifytiming constraints between edges

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 6/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Quantitative Timing DiagramsEdges and State Expressions

• Edges types

• Event edges: exact number of samples that satisfy the expression• Conditional edges: multiple consecutive sample values

• State expressions

• All evaluable relational expressions for the diagram and trace• State expression grammar in EBNF• Keywords for enhanced convenience and expressiveness

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 7/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Quantitative Timing DiagramsAssignments

• Value assignments, usable on all diagram elements

• Waveforms, edges, relationships, diagram

• Denoted as Element : Assignment similar to [FSS02]

• Evaluated when the de�ning element is evaluated to TRUE

• Assignment expression grammar for complex value calculations

• Evaluated assignments have

• Unique name• Assigned value• Trace time stamp of evaluation

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 8/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Quantitative Timing DiagramsEvaluation

• Evaluation - Advancing cutlines

1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value

• Success• All waveforms are traversed until their �nal states• No timing constraints were violated

• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Quantitative Timing DiagramsEvaluation

• Evaluation - Advancing cutlines

1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value

• Success• All waveforms are traversed until their �nal states• No timing constraints were violated

• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

OverviewQuantitative ExtensionsEvaluation

Quantitative Timing DiagramsEvaluation

• Evaluation - Advancing cutlines

1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value

• Success• All waveforms are traversed until their �nal states• No timing constraints were violated

• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Timing Diagram Editor

• Python-implementation of diagram editor and evaluation engine

• Detailed XML result �les for all calculated values and timings

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 10/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Integration with TRACE-CHECK

• TRACE-CHECK1 also supports veri�cation of properties formulatedin temporal logic (MTL) and via Python-implemented scripts

• Evaluation operates on traces with non-equidistant time stamps

1http://www.trace-check.de

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 11/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Integration with TRACE-CHECKReport View

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 12/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Lessons learnedUse Cases from the Automotive Domain

• Test engineers highly appreciate timing diagram speci�cation

• They often struggle with translating requirements into TL• In logic complex timing relationships between signal curvesunavoidably lead to complicated, strongly nested expressions

• TD nicely complement temporal logic

• Focus on the 'good-cases' or 'positive patterns' of speci�cation• TL: easier to formulate that something must not happen• QTD: testify whether a �nite trace segment behaves like expectedand give detailed information about the execution conditions

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 13/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Lessons learnedUse Cases from the Automotive Domain

• Test engineers highly appreciate timing diagram speci�cation

• They often struggle with translating requirements into TL• In logic complex timing relationships between signal curvesunavoidably lead to complicated, strongly nested expressions

• TD nicely complement temporal logic

• Focus on the 'good-cases' or 'positive patterns' of speci�cation• TL: easier to formulate that something must not happen• QTD: testify whether a �nite trace segment behaves like expectedand give detailed information about the execution conditions

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 13/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

EditorTool IntegrationVisualization

Visualization Prototype

• Aggregate and overview analysis results by processing result �les

• Diagram activations are plotted along a horizontal timeline

• Zoom, �lter and access concrete timings and values

• Stack and overlay multiple analyses for comparison

• B Visualize system executions on a functional level

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 14/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

ConclusionReferences

Conclusion

Summary

• Extended timing diagrams as speci�cation language for quantitativetrace analysis

• Implementation of TD editor and evaluation engine

• Integration into industrial veri�cation tools

• First application results and prototype for result visualisation

Future Work

1 Give formal syntax and semantics for TD evaluation over �nite traces

2 Use continuous signal descriptions inside state expressions

3 Logically and hierarchically combine multiple diagrams

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 15/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

ConclusionReferences

Conclusion

Summary

• Extended timing diagrams as speci�cation language for quantitativetrace analysis

• Implementation of TD editor and evaluation engine

• Integration into industrial veri�cation tools

• First application results and prototype for result visualisation

Future Work

1 Give formal syntax and semantics for TD evaluation over �nite traces

2 Use continuous signal descriptions inside state expressions

3 Logically and hierarchically combine multiple diagrams

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 15/ 16

IntroductionTiming Diagrams

ApplicationConclusion and Future Work

ConclusionReferences

References I

Werner Damm, Bernhard Josko, and Rainer Schlör.Speci�cation and validation methods.chapter Speci�cation and veri�cation of VHDL-based system-levelhardware designs, pages 331�409. Oxford University Press, Inc., NewYork, NY, USA, 1995.

Kathi Fisler.Timing diagrams: Formalization and algorithmic veri�cation.Journal of Logic, Language and Information, 8:323�361, 1999.10.1023/A:1008345113376.

Bernd Finkbeiner, Sriram Sankaranarayanan, and Henny B. Sipma.Collecting statistics over runtime executions.In Proc. of Runtime Veri�cation (RV02), pages 36�55. Elsevier,2002.

Andreas Richter Quantitative Trace Analysis using Timing Diagrams 16/ 16