ISTQB – Foundation Level

Fundamentals in Testing

Agenda• An introduction on ISTQB• ISTQB syllabus 2010• Fundamentals of Testing• Terms to Remember• Know your level

Introduction

ISTQB can be done :by everyone who are involved in

software testingwho wants a basic

understanding of software testing

Basic level for higher-level software testing qualification.

Syllabus• The new syllabus of version 2010 is

effective 30-Mar-2010• Learning objectives are classified as

follows:K1: remember, recognize, recall K2: understand, explain, give

reasons, compare, classify, categorize, give examples, summarize

K3: apply, useK4: analyze

Fundamentals of Testing (K2)

Course Objectives• Why is Testing Necessary? • What is Testing? • Seven Testing Principles • Fundamental Test Process • The Psychology of Testing

Why Testing is Necessary?

No software is defect free To learn about the reliability of the software Failures are expensive To gain customer satisfaction To sustain in business Requirement coverage

Why Testing is necessary?

Where Bugs Come From and What Bugs Do A human being can make an error (mistake), which produces a defect (fault, bug) in the code, in software or a system, or in a document. If a defect in code is executed, the system will fail to do what it should do (or do something it shouldn’t), causing a failure.

Error Fault( Bug or Defect) Failure

The Threat of Bugs– Company – Reputation for quality, unpredictable maintenance

cost, lack of confidence, lawsuits– People & Society – lost jobs, lost lives etc– Environment – Pollution ,Waste

Causes of Faults

Human error

Time pressure

Complexity in software requirements

Technology

Software Quality

• Testing and quality Tests gives confidence where they find few bugs Passing tests reduce the level of quality risk Failing tests provide a chance to improve quality The test set gives an assessment of quality When bugs are removed the quality is improved

• Testing should be a quality assurance activity

• “Fitness for Use” or “Conformance to requirements”

• Verification vs. Validation

How much testing is enough?

• Risks Technical Business Project

• Constraints Time Budget

The testing completion should done based on the above factors. Testing should provide sufficient information to stakeholders to make informed decisions about the release . To achieve it Prioritize your tests.

Exhaustive Testing is impossible

Terms to Remember

• Bug, Defect, Error, Failure, Fault, Mistake, Quality, Risk, Software, Testing, Risk, Exhaustive testing, root cause analysis

What is Testing?

Testing Objectives

Finding bugs and providing programmers with the information they need to fix important bugs

Gaining confidence about the level of quality of the system Preventing defects (through early involvement in reviews

and advanced test design) Provide information about the most important aspects of

the quality of the system under test Help management understand system quality

What is Testing?

Testing Process

Testing is a process rather than a single activity-there are a series of activities involved

All life cycle activities Both static and dynamic Test Planning and control Preparation Evaluation Software products and related work products

Phases of Testing• Unit/Component Test

– Find bugs in the individual pieces of the system under test before the pieces are fully integrated into the system

• Integration/String Test– Find bugs in the relationships and interfaces between pairs and groups

of components in the system under test as the pieces come together

• System Test– Find bugs in the overall and particular behaviors, functions, and

responses of the system under test as a whole

• Acceptance/Pilot Test– Demonstrate that the product is ready for deployment/release or to

assess quality and give information on the risk of deployment/release

• Maintenance Test– Check for errors introduced during development of the changes

• Operational Test– Assess non-functional system characteristics such as reliability or

availability

Testing vs. Debugging

Testing finds failures that are caused by bugs Debugging - The process of finding, analyzing and

removing the causes of failures in software …identifies the root cause of a bug… …repairs the code… …and checks that the defect is fixed correctly

Confirmation testing ensures the fix resolves the observed failure  

Different responsibilities: Testers test Programmers debug

Find Debug Confirm

Terms to Remember

• Code, Debugging, Development, Requirement, review, Test Case, Test Basis, Testing, Test Objectives.

Testing Principles• Principle 1 – Testing shows presence of defects

Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.

• Principle 2 – Exhaustive testing is impossibleTesting everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts.

• Principle 3 – Early testingTesting activities should start as early as possible in the software or system development life cycle, and should be focused on defined objectives.

• Principle 4 – Defect clusteringA small number of modules contain most of the defects discovered during pre-release testing, or show the most operational failures.

• Principle 5 – Pesticide paradoxIf the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this “pesticide paradox”, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

• Principle 6 – Testing is context dependentTesting is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.

• Principle 7 – Absence-of-errors fallacyFinding and fixing defects does not help if the system built is unusable and does not fulfill the users’ needs and expectations.

Terms to Remember

• Exhaustive Testing, Defect Clustering, Pesticide Paradox, Error Fallacy.

Test Process

Test Process - Plan, prepare, perform, perfect

• Planning and control • Analysis and design • Implementation and execution • Evaluating test exit criteria and reporting • Test closure activities

Planning & Control

Planning• Determine test scope,

risks, objectives, strategies• Determine required test

resources • Implement the test

strategies• Schedule test analysis and

design • Schedule implementation,

execution and evaluation of tests

• Determine the test exit criteria

Control• Measure and analyze

results• Monitor and document

progress, coverage and test exit criteria

• Initiate corrective actions • Make decisions

Analysis & Design

Analysis• Review the test basis (e.g.,

requirements or design specifications, network/ system architecture, quality risks)

• Identify test conditions, test requirements, or test objectives and required test data based on analysis of test items, its specification, behavior and structure

Design• Select specific

combinations of test data, actions, and expected results to cover the test basis

• Evaluate testability of the requirements and system

• Design the test environment

• Identify any required infrastructure and tools

Implementation & Execution

Implementation• Review the test basis (e.g.,

requirements or design specifications, network/ system architecture, quality risks)

• Identify test conditions, test requirements, or test objectives and required test data based on analysis of test items, its specification, behavior and structure

Execution• Select specific

combinations of test data, actions, and expected results to cover the test basis

• Evaluate testability of the requirements and system

• Design the test environment

• Identify any required infrastructure and tools

Exit Criteria, Reporting &Closure

Exit Criteria and Reporting• Check test logs against the

test exit criteria specified in test planning

• Assess if more tests are needed or if the exit criteria specified should be changed

• Write a test summary report for stakeholders

Closure• Confirm test deliverables,

final resolution or deferral of bug reports, and the acceptance of the system

• Finalize and archive test ware, test environment and test infrastructure

• Deliver test ware to the maintenance organization

• Perform a retrospective to capture improvements for future releases, projects, and test processes

Terms to Remember

• Confirmation testing, exit criteria, incident, regression testing, test basis, test condition, test coverage, test data, test execution, test log, test plan, test strategy, test summary report, test ware.

Psychology of Testing

How people influence testing success

The right mix of self-testing and independent testing

Communication between testers and others on the project team

Defining Tester Skills

• Testers are sometimes on receiving end of emotions brought on by news of project problems

• The Testers skills are defined as:– Reading specifications, emails, test cases, etc– Writing test cases, bug reports, test

documentation etc– Not ‘’Native Language” dependent– Pertinent technology, project and testing skills

Balanced Skills

• Effective and efficient tester have right mix of skills for tasks and activities

• Application domain expert– Understands intended behavior

• Skilled tester– Knows quality risks and test techniques

• Technical guru – Aware of technical issues and limitations

Domain

Expert

Testing skill

Technical

Guru

Levels of independence

Several levels of independence based ranking from lowest to highest:

Tests by the person who wrote the item under test; Tests by another person within the same team, such as

another programmer; Tests by a person from a different organizational group,

such as an independent test team; Tests designed by a person from a different organization or

company, such as outsourced testing or certification by an external body.

Effective CommunicationCommunicate findings on the product in a neutral, fact-focused way without criticizing the person who created it.

Don’t gloat, Don’t blame, be constructively critical and discuss the defect

Explain that by knowing about this now we can work round it or fix it so the delivered system is better for the customer.

Say what you liked, give praise as well as criticism, Show the risk honestly.

Start with collaboration rather than battles. Remind everyone of the common goal of better quality systems.

Be polite and helpful, Offer your work to be reviewed, Confirm the other person understanding

Terms to Remember

• Independent Testing

Exam Pattern• No of questions : 40• Passing Criteria : 60% or 25 questions to be

correct• Break down of questions : 7 questions from

Chapter 1

• Question Pattern:

Which statement regarding testing is correct? (Stem) a) Testing is planning, specifying and executing a program with the

aim of finding defects (key) b) Testing is the process of correcting defects identified in a

developed program (distracter) c) Testing is to localize, analyze and correct the direct defect cause

(distracter) d) Testing is independently reviewing a system against its

requirements (distracter)

All the Best !!