2011 05-27-icse

© ABB Group April 9, 2023 | Slide 1

Q-ImPrESSAn Industrial Case Study on Quality Impact Prediction for Evolving Service-oriented Software

Heiko Koziolek, ABB Corporate Research, GermanyBastian Schlich, Carlos Bilich, Roland Weiss, Steffen Becker, Klaus Krogmann, Mircea Trifu, Raffaela Mirandola, Anne Koziolek

Industrial automation: Process Control Systems


MotivationRelease History for an ABB Process Control System


2004 2005 2006 2007 2008 2009 2010

Version A First version release with complete system

concept Single environment from independent solutions Outstanding Operations Offering Function based Engineering Redundant Controllers and I/O capabilities Connectivity for Harmony and Melody FF, Redundant Profibus, HART, ABB Drives

Version B Increased system size SIL 2 Integrated Safety Connectivity for DCI and MOD 300 Alarm and Event Improvements Remote Clients via MS Terminal

Services

Version C Online Upgrade Capability Multi-User / Distributed Engineering Large screen / Multi-screen

enhancements Digital Security Improvements

Version C1 Increased system

size Multi-system

Integration SPI Integration

(PETI) MODBUS TCP

Version C2 Virtualization support MS WPF Graphics SIL3 Safety IEC 61850 (Intel Elect Devices) New PM866 controller (2x PM864) New S800 I/O (non-red HART) New Power Supplies with smaller

footprint Evolution Libraries MOD300 and Infi90

Version C3 Windows 7 support Alarm Analysis and Alarm

Shelving WirelessHART Integration Profinet, Ethernet IP, DeviceNet New Controller PM891 (2x

PM866) Engineering efficiency

improvements Detailed difference reporting Foundation Fieldbus

improvements

MotivationProblems of software evolution at ABB

Continuous evolution of ABB software systems

New requirements, technologies, failure reports

Software maintenance and evolutionare a large cost factor for ABB software

development

Current practice

Experience to rationalize design decisions

Prototyping for new technologies, performance impacts

Unknown change impacts on performance/reliability

Apply model-based prediction methodsfor systematic decision supportto save costs and achieve higher quality?


Q-ImPrESS MethodOverview

Q-ImPrESS Workbench

Case Study


Manual ModelingSteps executed to create a Q-ImPrESS model

Modelling static structure• Analyzed architectural documenation• Identified four key use cases• Abstraction level: process = component

Modelling dynamic structure

• Created testbed, installed system• Recorded component transitions• Derived transition probabilities

Validating the model• Created Q-ImPrESS model in workbench• Applied Q-ImPrESS consistency checker• Discussed the model with architects


~2.5 person months

Manual ModelingQ-ImPrESS model of the ABB process control system


Performance PredictionSteps executed to determined resource demands

Preparing measurements• Abstracted memory, virtualization

overheads• Designed experiment runs per use case

Measuring performance• Used Windows Performance Monitor• Determined >20 CPU and HD demands

Predicting performance• Annotated Q-ImPrESS model• Executed SimCom & LQN solver• Modelled alternatives and ran predictions


~1 person month

Performance PredictionSample predictions for different design alternatives


• Achieved prediction error below 30 percent• Easy to analyze different evolution scenarios

Pro

• Data collection consumed more time than expected• Many bottlenecks below the architectural level

Con

Performance PredictionResults: Measurements vs. Simulation Results


Workload PerfMonMeasured

SimuComPrediction

Error (%) LQNSPrediction

Error (%)

30 17.146 12.467 27.288 12.464 27.305

60 26.681 22.366 16.174 22.343 16.260

90 31.902 32.347 1.395 32.322 1.317

120 39.016 42.432 8.754 42.329 8.490

150 51.929 51.943 0.027 51.760 0.326

Reliability PredictionSteps executed to determine failure probabilities

Preparing measurements• Conducted literature search• Selected software reliability growth model

from IEEE 1633-2008 (Littlewood/Verrall)

Measuring reliability• Acquired access to bug tracking system• Conducted curve fitting per subsystem• Determined subsystem failure probabilities

Predicting reliability• Annotated Q-ImPrESS model• Executed PRISM model checker• Conducted sensitivity analysis


~1 person month

Each line shows how the system reliability changes if we change one subsystem reliability (8 subsystems in total)

Reliability PredictionSample sensitivity analysis


Reliability PredictionResults


• Identification of sensitive subsystems• Better planning of test budget distribution• Ability to analyze different usage profiles

Pro

• Abstraction level too high: (e.g., no hardware reliability)• Data collection for lower abstraction levels difficult• Validation difficult: would take long to get data

Con

More research and tool development needed

ConclusionsSummary

Q-ImPrESS

provides a structured method and useful tool support

is best used for evolutionary changes, not full redesigns

still needs to demonstrate costs/benefits


ConclusionsFuture Work

Future work desired by ABB:

More robust reverse engineering tools

Model transformations from UML to Q-ImPrESS

Tools and best practices for data collection


2011 05-27-icse

Technology

costsbenefits abb group

case study abb group

qimpress model abb group

data collection abb

simulation results abb

resource demands abb

abb corporate research

abb drivesversion c3