comp 587 parker li bobby kolski. automated testing tools assist software engineers to gauge the...
TRANSCRIPT
![Page 1: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/1.jpg)
Comp 587
Parker Li
Bobby Kolski
Metrics for Measuring the Effectiveness of Software-Testing Tools
![Page 2: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/2.jpg)
Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects of software-testing.
Automated testing tools vary in their:
• Underlying approach
• Quality
• Ease-of-use
How does a project manager choose the best suitable testing tool?
Introduction
![Page 3: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/3.jpg)
• The history of software metrics began with counting the number of lines of code (LOC).
• It was assumed that more lines of code implied more complex programs, which in turn were more likely to have errors.
• However, software metrics have evolved well beyond the simple measures introduced in the 1960s.
Software Quality Metrics
![Page 4: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/4.jpg)
• Since the first introduction of LOC, metrics for traditional or procedural source code have increased in number and complexity.
• Cyclomatic complexity
• e is the number of edges, n is the number of nodes and p is number of unconnected parts of G.
• Offers an estimate of the reliability, testability, and maintainability of a program, based on measuring the number of linearly independent paths through the program.
Procedural (Traditional) Software Metrics
![Page 5: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/5.jpg)
Function Point is a measure of the size of computer applications and the projects that build them.
Procedural (Traditional) Software Metrics (Cont.)
![Page 6: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/6.jpg)
• Weighted-methods-per-class (WMC) – the sum of the individual complexities of the methods within that class.
• Depth of inheritance tree (DIT) – defined as the maximum length from the node to the root of a class tree. Deeper a class is in the inheritance hierarchy, more complex it gets.
• Number of Children (NOC) – Immediate subclasses of class. Large NOC implies great amount of inheritance and resue.
• Coupling between object classes (CBO) – When a class inherits methods, instance variables, they are coupled. Greater number = complexity
Object-Oriented Software Metrics
![Page 7: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/7.jpg)
• Number of key classes (NKC)
• Number of subsystems (NSUB)
• Class size (CS)
• Number of operations added by a subclass (NOA)
• Average method size
• Average number of instance variables
• Class hierarchy nesting level
Object-Oriented Software Metrics
![Page 8: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/8.jpg)
• Does it offer critical support for planning tests and monitoring test progress?
• Does it have an analysis feature where it assesses the characteristics of the software quality?
• Does it offer guiding dynamic testing?
• How mature is the tool? Is it easy to implement for your system?
Which Software-Testing Tool to use?
![Page 9: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/9.jpg)
• Human Interface Design (HID) – Tools with well-designed human interfaces enable easy, efficient, and accurate setting of tool configuration.
• Tool Management (TM) – the tool should ensure proper management of information.
• Ease of Use (EU) – the tool should be easy to use by both new and familiar users.
• User Control (UC) – the tool should allow extensive and detail testing coverage rather than one bulky one.
Metrics for Tools that support testing procedure software
![Page 10: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/10.jpg)
3 Tools for Software Testing
•LDRA Testbed
•Parasoft Testbed
C++ Test
CodeWizard
Inspire++
•Telelogic Testbed
Logiscope
![Page 11: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/11.jpg)
• Coverage Report (Dynamic Analysis)
Branch Coverage
Statement Coverage
• Metrics
Cyclomatic Complexity (Static Analysis)
• Quality Report
LRDA Testbed
![Page 12: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/12.jpg)
LRDA Testbed – Branch/Decision coverage Part 1 (Part of dynamic testing)
![Page 13: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/13.jpg)
LRDA Testbed – Code/branch/decision coverage Part 2 (Part of dynamic testing)
![Page 14: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/14.jpg)
Parasoft Testbed
• C++ Test White-Box Black-Box Regression testing
• Code Wizard 170 industry-accepted C/C++ coding standards
•Inspire++ C/C++ runtime error checking
![Page 15: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/15.jpg)
Parasoft Testbed (metrics)
![Page 16: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/16.jpg)
Parasoft Testbed C++ Test Bug Detective (static analysis)
![Page 17: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/17.jpg)
• Logiscope
• TestChecker – mentioned in the article
• RuleChecker – mentioned in the article
• Audit *
• Reviewer *
* included in the current version of the tool and shown for completeness but not covered further in the slides.
Telelogic Testbed
![Page 18: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/18.jpg)
Telelogic Logiscope – Bought by IBM – The current version is Rational Telelogic Logiscope
![Page 19: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/19.jpg)
• Human Interface Design (HID) Comparison
• Keyboard-to-mouse switches (KMS)
• Input fields per functions (IFPF)
• Average length of input fields (ALIF)
• Button recognition factor (BR)
Exercising the Software-Testing Tools (Comparison of the tools themselves)
![Page 20: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/20.jpg)
Test case generation (TCG)
Automated test case generation (ATG)
Test case reuse functionality (TRF)
Special note: LDRA does not
automatically generate test
cases but does provide
user-friendly features such
as pull-down menus for created
test cases therefore it was
assigned an eight for its level of ATG.
Exercising the Software-Testing Tools (Comparison of the tools themselves)
![Page 21: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/21.jpg)
LRDA – For Object Oriented tests there were issues with the tool, so no results were obtained.
High levels of knots indicate that the code is disjointed
LDRA appends the Motor Industry Software Reliability Association (MISRA) C Standard and the Federal Aviation Authority’s DO-178B standard.
Analysis of Results (The output of the tools)
![Page 22: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/22.jpg)
• Software Metrics have come a long way since the LOC era.
• Software Metrics can be use not only to measure the quality of software, but also the effectiveness of the software testing tool.
• Choosing a software testing tool is not simple.
Importance of this article
![Page 23: Comp 587 Parker Li Bobby Kolski. Automated testing tools assist software engineers to gauge the quality of software by automating the mechanical aspects](https://reader036.vdocuments.site/reader036/viewer/2022070323/56649d985503460f94a81c18/html5/thumbnails/23.jpg)
Questions?