tcs2411 software engineering1 software quality assurance “what is quality software and how to...

TCS2411 Software Engineering 1

Software Quality Assurance

“What is quality software and how to achieve it?”


Lecture ObjectivesTo discuss what is meant by quality in

softwareTo understand the importance of quality in

software engineeringTo describe the features of quality softwareTo describe the measures used in

assessing software qualityTo understand the international standards

that are applied to assess software quality


What Is Quality Software?

Is it software that works?Is it software that is error-free?Is it software that is very well-

documented and has help facility for all its functions?

Is it software that does not ‘hang’ the machine?

Is it software that does not ‘bomb’?


Group Project

How do you ensure the quality of the software you are developing?

Have you defined the standards and tasks to ensure the quality of the software?

Will the customer say that your software is a quality product?

Are you fulfilling the customer’s expectations?


Software Quality Definition

Conformance to Explicitly stated functional and

performance requirementsExplicitly documented development

standards, andImplicit characteristics that are

expected of all professionally developed software

(Pressman)


Software Quality Factors

Low defects level when deployed Zero defect most preferably

High reliability Capability of running without crashes

Majority of the user satisfy with software when conducted the survey

Effective customer supportRapid defect repair


Software Qualities

Correctness Reliability Robustness Performance User friendliness Verifiability Maintainability Reparability Safety

Evolvability Reusability Portability Understandability Productivity Size Timeliness Visibility

Select the critical attributes and plan how to achieve them


Benefits Of Software Quality

Reduced maintenance costStable and useful productSatisfy customer needsBetter chances for continuing

releasesBuild corporate culture and identityBetter chances for software and

design reuse



Quality assurance – a set of establish framework (procedures & standard) used by an organization to achieve high quality software

Essential activity for any business that produce product to used by others It need to be planned and systematic It does not just happen

Assure each of the software qualities is met Goal set in requirements specification Goal realized in the implementation

SQA is applied at every stage of the software process Quality standard is selected to apply to the software

process


Why SQA Payoff ?

logscale

cost to find and fix a defect

1

10

100

Reqmts

1.00

Design

1.30

Code

2.00

Test

4.00

Systemtest

13.00

Fielduse

80-130

SQA pays off here becausedefects are cheap to fix


Principle Of SQA

1. Set the standard and quality attributes that a software product must meet The goal to achieve

2. Measure the quality of software product There is a way to determine how well the

product conforms to the standards and quality attributes

3. Track the values of the quality attributes It is possible to assess how well we are doing

4. Use the information of software quality to improve the quality of future software product There is a feedback into the software

development process


SQA Activities

Application of Technical MethodsConduct of Formal Technical ReviewsSoftware TestingEnforcement of StandardsControl of ChangeMeasurementRecord keeping and Reporting


Application of Technical Methods

Select appropriate tools and methods to capture system requirements, analyze system, design, implementation and testing

Why use methods and tools To ensure high quality is achieve Analyst achieve high quality specifications Designer develop high quality designs Ability to measure quality in specifications and

designs


Conduct of Formal Technical Reviews

FTR is an activity to assess qualityA stylised meeting conducted by

technical staff with the sole purpose of uncovering quality problems

Found to be effective in uncovering defects in software

All major software items should be subjected to technical review


Software Testing

Testing is a process of executing a program with the intent of finding errors, locate errors and to proof system correctness

Testing is conducted based on the developed test cases Capture actual output Compare actual output with expected output

Actual == Expected : Test case succeedActual != Expected : Test case failed

All test cases result must be recorded


Software Testing(Cont)

To identify errors in software developedTest case design methods produces

tests to be used on the softwareDevelopment of test strategy for a set

of tests that uncovers all possible errorsThorough testing is not as effective as

expected in most cases


Enforcement of Standards

Formal standards and procedures varies from company to company

Could be dictated by customer, regulations, or self-imposed

If formal (written) standards exist, they must be followed to ensure quality

Assessment of compliance is done through FTR or audit


Control of Change

An activity executed throughout the system life cycle to control change of products and life cycle artifacts

Items that needs control changes Plans Specification Procedures Audit Report Support

A software library need to be construct that store, manages and track these items

•Need to be identified and described•Need to be uniquely itemized•Need to be organized


Control of Change (Cont)

Every change has potential for introducing errors or creating side effects that propagate errors

Change control process ensures quality

Formalizing requests for change, evaluating nature of change, and controlling the impact of change


Measurement

Measurement : Numeric derivation for some attribute of a software product / process Compare the value with standard apply Often expressed in $$$ and days Integral part to any engineering discipline

Software metrics must be collected to track quality

Also to assess the impact of methodological and procedural changes on improved software quality


Record keeping and Reporting

Historical record for project - results of reviews, audits, change control, testing, other SQA activities

Provide procedures for collection and dissemination of SQA information

Dissemination to development staff based on need-to-know basis


McCall Software Quality Model

ProductRevision

ProductTransition

Product Operations


McCall’s Software Quality Factors

Product Operations Operational characteristics

Product Revision Ability to undergo changes

Product Transition Adaptability to new environments


McCall’s Software Quality Factors (Cont)

Factor Criteria Description

Product Revision

Maintainability

Can I fix it?

Flexibility Can I change it?

Testability Can I test it?

Product Transition

Portability Will I be able to use it on another machine?

Reusability Will I be able to reuse some of the other software in other application?

Interoperability

Will I be able to interface it with another system?

Product Operation

Correctness Does it do what I want?

Reliability Does it do it accurately all the time?

Efficiency Will it run as well as it can?

Integrity Is it secure?

Usability Is it easy to use?


FCM By McCall Approach

Simplicity

Reliability

S1S2 S3

Factors – user concern oriented

Criteria – software concern oriented

Metrics – quantitative measures


Measuring the Factors

The factors are difficult to measure directly McCall defined metrics that are assessed

subjectively, on a scale of 0 to 10 Includes auditability, accuracy, completeness,

consistency,modularity, etc. Factors calculated as follows

Fq = c1 x m1 + c2 x m2 + … + cn x mn

where

cn = Weighting coefficient (or importance factor)

mn = Criteria measured


Alternative Measurements

Hewlett-Packard developed a set of quality factors - FURPS Functionality - features, capabilities, overall Usability - human factors, consistency, doc. Reliability - failure, accuracy, MTBF, recovery Performance -speed,response time, efficiency Supportability - maintainability, testability

Quality metrics for each step in SE process


Measuring Quality Factors In SE Process

Investigations/Specifications Design Implementation Testing Support

F

U

R

P

S

# .....% ...



Quality assurance - essential activity for any business that produces products to be used by others

SQA - a planned and systematic pattern of actions to ensure quality in software

“Quality is Job #1”


Software Reviewlike a “filter” for SE processmust be applied at various points

during developmentall technical work need reviewingalso checks one’s mistakeshelps to discover defects early

design can introduce 50 - 65% of errors review can uncover 75% of them cost of subsequent steps reduced


ISO 9000

Set of standards that can be applied to a range of organisations from manufacturing through to service industries

ISO 9001 - general standards, applies to design, development, maintaining products

ISO 9000-3 - supporting document for software development

Various countries also have own standards


ISO Certification

Some countries have bodies which certify that the quality process in an organisation conforms to ISO requirements

In Malaysia, SIRIM performs this functionCustomer looks for certification in an

organisation to indicate the quality of products or services


Areas covered by ISO 9001/9000-3 STANDARD (1/3)

A. A. Quality System framework1. Management responsibility

quality policy defined, documented, understood, implemented and maintained

responsibilities and authorities for all personnel defined in-house verification resources defined, trained and funded a designated manager ensures that the quality program is

implemented and maintained

2. Quality system requires a documented quality system be established should be an integral process throughout the entire life cycle

3. Internal quality system audits requires audits be planned and performed results communicated to management deficiencies corrected

4. Corrective action requires causes of non-conforming product be identified potential causes of non-conformance eliminated procedures changed as a result



B. Quality Life Cycle Activities

1. Contract review contracts are reviewed to

determine whether the requirements are adequately defined, agree with the bid, and can be implemented

2. Purchaser’s requirements specification procedures to identify and

collect purchaser’s requirements

3. Development planning production processes are

defined and planned4. Quality record

quality assurance process is planned

5. Design and implementation procedures to control and

verify the design

includes planning designactivities, identifying inputs and outputs, verifying the design and controlling design changes

6. Testing and validation requires systems/products to be

tested and validated before use records of testing are kept

7. Acceptance procedures for acceptance

should be established8. Replication, delivery and installation

procedures for the above should be established and maintained

9. Maintenance requires that maintenance

activities be performed as specified



5. Rules, practices and conventions are in place and followed

6. Tools and Techniques are applied appropriately

to support the development process

7. Purchasing purchased products

conform to their specified requirements

8. Included software product should be verified and

maintained9. Training

training needs should be identified and training provided since related tasks may require qualified personnel

training records should be maintained

1. Configuration management process of development

and maintenance should be documented and controlled

2. Document control distribution and

modification of documents should be controlled

3. Quality records quality records should be

collected, maintained and dispositioned

4. Measurement where appropriate,

adequate statistical techniques should be identified and used to verify the acceptability of process capability and product characteristics

C. Quality System Supporting Activities


Quality Management System

ISO 9000Quality Models

OrganisationalQuality Manual

Project 1Quality Plan



OrganisationalQuality Process

Project QualityManagement

instantiated as

documents

is used to develop instantiated as


ISO & SQA

An organisation that have SQA tasks in place and a quality management process would have most of ISO elements

Ideally quality management should be separate from project management

This is to ensure quality considerations are not compromised by concerns of budget and schedule


Process Improvement

Understanding existing processes and changing these processes to improve product quality and/or to reduce costs and development time

Assume that key factor to product quality is the quality of development process

kaizen - process of continuous process improvement


Capability Maturity Model

Developed by Software Engineering Institute (SEI) of Carnegie-Mellon U

SEI was established to improve the capabilities of US software industry

Focus on improving the process of software engineering

Similar to ISO, organisations are assessed and certified to be on Level 1 to Level 5


Process Maturity Model

Level 1InitialLevel 1Initial

Level 2Repeatable

Level 2Repeatable

Level 3DefinedLevel 3Defined

Level 4Managed

Level 4Managed

Level 5Optimizing

Level 5Optimizing


Level 1 Initial

No effective management procedures or project plans

No organisational mechanisms to ensure consistent use of formal procedures

Successful software depends more on individual skill rather than organisation


Level 2 Repeatable

Formal management, quality assurance and configuration control procedures

Organisation can successfully repeat projects of the same type

Lack of formal process model


Level 3 Defined

Organisation has defined its process and has a basis for qualitative process improvement

Formal procedures are in place to ensure that the defined process is followed in all software projects


Level 4 Managed

Organisation has a defined process and a formal programme of quantitative data collection

Process and product metrics are collected and fed into the process improvement activity


Level 5 Optimising

Organisation is committed to continuous process improvement

Process improvement is budgeted and planned and is an integral part of the organisation’s process


Key Process Area

InitialInitial

RepeatableSoftware Configuration ManagementSoftware Quality AssuranceSoftware Subcontract managementSoftware project tracking and oversightSoftware project planningRequirements management

RepeatableSoftware Configuration ManagementSoftware Quality AssuranceSoftware Subcontract managementSoftware project tracking and oversightSoftware project planningRequirements management

DefinedPeer reviewsIntergroup coordinationSoftware product engineeringIntegrated software managementTraining programOrganization process definitionOrganization process focus

DefinedPeer reviewsIntergroup coordinationSoftware product engineeringIntegrated software managementTraining programOrganization process definitionOrganization process focus

ManagedSoftware quality managementQuantitative process management

ManagedSoftware quality managementQuantitative process management

OptimizingProcess change managementTechnology change managementDefect prevention

OptimizingProcess change managementTechnology change managementDefect prevention


Key Process AreasLevel 1 Initial

noneLevel 2 Repeatable

Software configuration management Software quality assurance Software subcontract management Software project tracking and oversight Software project planning Requirements management


Key Process Areas (Continued)

Level 3 Defined Peer reviews Intergroup coordination Software product engineering Integrated software management Training programme Organisation process definition Organisation process focus


Key Process Areas (Continued)

Level 4 Managed Software quality management Quantitative process management

Level 5 Optimising Process change management Technology change management Defect prevention


References

“Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001

“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001

tcs2411 software engineering1 software quality assurance “what is quality software and how to...

Documents

software quality slide

software quality zto

software process slide

software pressman slide

tcs2411 software engineering10

tcs2411 software engineering3

software engineering

quality product