Software Quality Engineering

Chapters 1-3

Overview, Software Quality and Quality Assurance

Ch.1: Overview and Basics

General Book Information Quality: View/Measure/Model QA Activities/Alternatives From QA to SQE

Ch.1: Main Problems Addressed

Deliver software system that... does what it is supposed to do. does the things correctly. respectively validate and verify software system to

show/demonstrate/prove it (“does").

Major difficulties for the above: Size: MLOC products common Complexity Environmental stress/constraints Flexibility/adaptability expected no silver bullet", but...SQE (software quality engineering)

helps

Ch.1: SQE as Answer

Major SQE activities: Testing: remove defect & ensure quality Other QA alternatives to testing How do you know: analysis & modeling

Scope and content hierarchy: Fig.1.1

Ch.1: Book Contents

QA alternatives/activities: Testing (Part II) Other alternatives (Part III):

defect prevention (Ch.13) inspection, review, analysis (Ch.14) formal verification (Ch.15) defect containment (Ch.16)

Analysis and improvement (Part IV) Issues in different QA alternative

Applicability and effectiveness Dealing with quality problems/defects:

prevention/removal/tolerance Cost Improvement (Part IV).

Ch.2: General Quality Views

In Kitchenham & Pfleeger (1996): Transcendental view: hard to define, but recognized User view: fitness for purpose or meet user’s needs Manufacturing view: conform to process standards Product view: inherent product characteristics. Value-based view: customers’ willingness to pay.

In Prahalad & Krishnan (1999): Quality defined by Conformance/adaptability/innovation Traditional: conformance only Evolved to include service, manage expectations:

0 defects -> 0 defections in customers Domain specific (for info age?):

specificity, stability, evolvability

-ilities User quality expectations

Performs right functions (correctness) reliability usability Install-ability inter-operability adaptability cost

Producer quality expectations fulfill contractual obligations Modifiability (flexibility) maintainability portability profitability

Ch.2: Quality Frameworks

Focus in various frameworks/mega-models different McCall: factors, criteria, and metrics Basili: GQM (goal-question-metric) SEI/CMM: process / levels of maturity ISO 9000: series of standards Dromey: component reflects quality-attributes

ISO 9126 top-level quality characteristics (pg. 18): Functionality: what is needed? Reliability: function correctly. Usability: effort to use. Efficiency: resources needed. Maintainability: correct/improve/adapt. Portability: one environment to another.

Alternative frameworks: company-specific, web-based, etc.

Ch.2: Defining Quality

Quality: views and attributes

SQE focus: correctness-related

Ch.2: Defect and Quality

Defect definition – a problem with the software, either with its external behavior or internal characteristics Failure: external behavior

deviation from expected behavior as specified Fault: internal characteristics

cause for failures Error: incorrect/missing human action

conceptual mistakes Bug/debug: avoid problematic terms, use instead defect

detection and removal Causal relations (not necessarily 1-1):

errors => faults => failures: See Fig 2.1 (p.21) Defect handling/resolution: Chapter 4.

Ch.3: QA: Dealing with Defects

QA: quality assurance Focus on correctness aspect of quality QA as dealing with defects. Many activities: testing & others How QA alternatives deal with defects =>

classification of QA alternatives Classification schemes to deal with defects:

Prevention – error blocking or error source removal Removal (detect them first) Containment

Classification illustrated: Fig 3.1 (p.30)

Ch.3: Error/Fault/Failure & QA

Preventing fault injection Causal/statistical/etc. analyses based Preventive measures:

education, technology, process, tools

Formal methods: formal verification (faults absent)

Removal of faults Inspection: faults discovered Testing: failures trace back to faults

Tolerance of faults Local fault does not imply global failure Dynamic containment measures to tolerate faults

Ch.3: Defect Prevention Overview

Error blocking Error: missing/incorrect actions Direct intervention Error blocked => fault injections prevented Rely on technology/tools/etc.

Error source removal Root cause analysis =>identify error sources Removal through education/training/etc.

Details: Chapter 13.

Ch.3: Defect Prevention: Formal Verification Overview Motivation

Fault present: revealed through testing/inspection/etc.

Fault absent: formally verify. Basic ideas

Behavior formally specified: pre/post conditions, or as mathematical functions.

Verify “correctness": intermediate states/steps, axioms and compositional rules.

Approaches: axiomatic/functional/etc.

Details: Chapter 15.

Ch3: Other Defect Prevention Techniques Appropriate use of software methodologies

and technologies Use of information hiding principle and

abstraction and modularization Better managed processes Enforcement of standards Use of tools

Ch.3: Defect Reduction: Inspection Overview Artifacts (code/design/test-cases/etc.) from

requirements/design/coding/testing/etc. phases. Informal reviews:

Self conducted reviews. Independent reviews. Orthogonality of views desirable.

Formal inspections: Fagan inspection and variations. Process and structure. Individual vs. group inspections. What/how to check: techniques .

Details: Chapter 14.

Ch.3: Defect Reduction: Testing Overview

Product/Process characteristics: Object: product type, language, etc. Scale/order: unit, component, system, … Who: self, independent, 3rd party

What to check: Verification vs validation External specifications (black-box) Internal implementation (white/clear-box)

Criteria: when to stop? Coverage of specs/structures. Reliability => usage-based testing techniques

Much, much more in Part II.

Ch.3: Defect Containment: Fault Tolerance Overview Motivation

Fault present but removal infeasible/impractical Fault tolerance => contain defects

FT techniques: break fault-failure link Recovery: rollback and redo NVP: N-version programming

fault blocked/out-voted

Details: Chapter 16.

Need to know these for future questions and Master’s exam What are 11 major software quality factors?

What are seven software quality assurance activities?

What is the difference between verification and validation?

11 Major Quality Factors Correctness – meets specs and objectives Reliability – perform intended function

with required precision (includes robustness)

Efficiency - amount of code and resources Integrity – controlled access Usability – effort required to learn/operate

Quality Factors cont. Maintainability – ease locate/fix error Flexibility (modifiability) – ease to modify

operational program Testability – ease to test Portability – works in other environments Interoperability – works with other systems Reusability – ease of reuse

7 Activities Application of technical methods and tools (to

ensure QA) Formal technical reviews Software testing Enforcement of standards to process Control of change (configuration

management) Measurement (product and process) Record keeping and reporting (information for

reviews, etc.)

Discussion Questions What is essential (but not sufficient) to earn the assessment

that the software is dependable? _____________ Other than that, what should we put greater emphasis on to attain high quality software?________________

Which of the 11 software quality factors concern product revision, which concern product operation and which concern product transition?

revision transition operation

Article – Fill in Figure 1.