introduction to automated quality assurance

(1)

Automated Quality Assurance

Philip Johnson

Collaborative Software Development Laboratory

Information and Computer Sciences

University of Hawaii

Honolulu HI 96822

(2)

Objectives Understand motivation for automated quality assurance through static analysis, and how it differs from “manual” quality assurance.

Learn about various automated quality assurance tools.

Understand the strengths and weaknesses of the Java-based static analysis tools used in this class (Checkstyle, PMD, FindBugs).

(3)

Quality Assurance High level:•Does the system satisfy the three prime directives?

Low level:•Does the system have “good” tests?•Does the system conform to coding standards?•Does the system contain code known to be defect-

prone?•Does the system satisfy its requirements?

We use a combination of “manual” and “automated” techniques to assess QA at the “high” and “low” levels.

(4)

Manual QA Examples of manual QA techniques:•Writing unit tests with JUnit•Conducting code reviews.

Strengths of manual QA:•Find defects involving requirements.•Low false positive rate.

Weaknesses of manual QA:•Difficult/expensive approach to low-level

implementation defects. •Must be redone for all projects.

(5)

Automated QA Examples of automated QA:•Lint, Checkstyle, PMD, FindBugs, DependencyFinder,

Coverity, FxCop

Strengths of automated QA:•Can find classes of implementation defects that

developers may not be skilled enough in the language/environment to detect via testing or inspection!•Can be reused on any project.

Weaknesses:•Potential high false positive rate. •Does not find requirements-level defects.

(6)

A Quick Tour of Automated QA Tools

(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

(15)

(16)

(17)

(18)

(19)

The three primary Automated QA tools for this class

(20)

(21)

Checkstyle Checkstyle performs source code analysis.•Originally for "coding standard" (formatting)•Now includes design-level best practice compliances.

Classes of checks:•JavaDoc, Naming Conventions, Headers, Size Violations, Imports, WhiteSpace, Modifers, Block Checks, Coding, Class Design, Duplicate Code, Metrics, J2EE.

Can be extended with new checks. Use a configuration file to customize what checks your system should comply with.

(22)

(23)

(24)

PMD

PMD also performs source code analysis.•More 'design' oriented than Checkstyle.•Lots of overlap.

PMD rulesets:•Basic, Braces, Code Size, Clone, Controversial, Coupling, Design, Finalizers, Import, J2EE, JavaBeans, JUnit, Logging, Migrating, Naming, Optimizations, Exceptions, Strings, Security, Unused Code, JSP, JSF.

Eventually, can choose one of PMD/Checkstyle.

(25)

(26)

(27)

FindBugs

FindBugs analyzes byte codes, not source codes.

Example problem: ignored return value:•String b = "bob"; b.replace('b', 'p'); if(b.equals("pop")){...}

FindBugs defect detectors are quite different in nature from source code analyzers because the system can perform forward and backward data flow.

(28)

(29)

QA in this class One goal of this course is to provide you with a set of techniques to build•high quality systems•as fast as possible

To accomplish this, you must learn how to apply the correct mix of automated and manual QA techniques. •Don’t write tests (or do reviews) to catch defects that an automated QA tool can find.•Don’t believe that automated QS tools can find all of your errors.

(30)