software testing for
TRANSCRIPT
Software Testing for Safety-Critical Applications
Presented by: Ciro Espinosa & Daniel Llauger
Agenda
Introduction Software and Industrial Automation Software Testing Strategies
Black-Box Testing White-Box Testing
Software Testing Processes and Steps Unit Testing Integration Testing
Top-Down Integration Testing Bottom-Up Integration Testing
System Testing Software Performance Testing
Availability, Survivability, Flexibility, Stress, Security, Usability and Maintainability Testing
Software Maintenance Conclusion
Introduction
Safety-critical applications in industrial automation have unique concerns for software quality (demanding requirements on system performance).
Software testing consists in functional and performance testing.
Software and Industrial Automation
The possibility for malfunction of complicated industrial automation software becomes much greater than the traditional one.
Embedded measurement and control systems in life-critical systems have unique concerns for system safety and reliability.
Any error in the software system is fatal to the real-world applications of industrial automation software
Basic testing principles:
Present expected testing results when designing test cases.
Separate sw testing team from sw development team.
Design invalid test cases.
Perform regression testing each time sw under-test is revised.
Tester should concentrate on error-prone program segments.
Software Testing Strategies
Black-Box Testing
White-Box Testing
In practical applications, the tester often combines black-box testing and white-box testing to conduct a more thorough test of the software.
Software Testing Processes and Steps
Testing real-time & embedded software is difficult
50 % of overall work in life cycle dedicated to testing and optimizing
Complexity – large number of potential paths
Real-time constraints exacerbate testing
Unit Testing
Integration Testing: Top-Down Integration
Bottom-Up Testing:
System testing
Find discrepancies between actual SW performance to its expected performance by comparing it w/ requirements
Test cases created based on requirement analysis description
On-site testing after SW installed with other equipment
Software Performance Testing
Real-time monitoring & control system Sensor measures monitored values Manner in which error between actual
& target measurement to be corrected determined by microprocessor-based controller
Actuator executes command to drive control variable close to target values.
Availability testing
Applies to time-critical, on-time, & real-time applications
Real-time SW needs to be designed to meet time constraints
Must meet deadline in responding to an operation
Control correctness and real-time restrictions can be considered simultaneously in design process
Availability testing (cont.)
Example: Chemical plant Alarm MGMT SW responsible for
triggering a siren (alarm) when abnormal process occurs
If it cannot perform alarm operation quickly, then disaster or production incident occurs
Reliability testing
Measures the likelihood for failure-free SW operations
Tests ability to operate w/o failure in environment designed for
Capture any abnormal status Why is this testing needed?
Low reliability SW can destroy industrial equipment or have a loss of life
To guarantee quality in manufactured products
Survivability Testing
Distributed and embedded real-time software system should be able to recover from the degraded performance.
Fault-tolerant control algorithms should be incorporated into the software design to maintain software performance in the presence of component failures.
Flexibility Testing
In the industrial automation area, the sw needs to work with heterogeneous hardware drivers and software components provided by different manufactures and vendors.
Today, open architecture-based software is gradually replacing traditional, proprietary software architecture
Stress Testing
Stress testing tests the software by pushing the system to its limits.
Stress testing may find both hardware and software defects.
Stress testing can be used to examine what types of system failures will occur when the system is heavily overloaded.
Security Testing
Security is an increasingly important issue in industrial automation software systems, especially with the proliferation of Internet-based industrial applications.
System security needs to be considered during all phases of the software life cycle.
Usability Testing
Usability tests how well the user operates the software system and likes doing so.
It is crucial to design the software with high usability
GUI design is an important component in system usability.
Maintainability Testing
High maintainability enables the released software to be revised in the presence of errors during system operations and makes the software expansion and change easy for new applications.
Maintainability should be considered from the very start of the life cycle.
For complex software, high maintainability becomes more necessary, as it is hard to identify the faulty lines of code without well-written documentation.
Software Maintenance
Establish safe & well controlled mechanism for update and modification
ID unexposed defects after installation Adapt to changing user requests & to
different operating conditions Corrective maintenance Adaptive maintenance Perfective maintenance Preventative maintenance
Conclusion
In the testing of industrial automation software systems, performance, flexibility, reliability, survivability, and usability should also be tested thoroughly.
Testing, code inspections and reviews are needed to prevent loss of life or industrial accidents
Questions?