CSTE - Skill Category 1 - 44 © Copyright 2014 / All rights reserved
What is a Process?
A process is a set of activities that represent the way work is performed. This work effort includes efforts of people and equipment guided by policies, standards, and procedures. The outcome from a process is usually a product or service.
CSTE - Skill Category 1 - 45 © Copyright 2014 / All rights reserved
What is a Process Defect?
A process is considered defective when the output of that process is outside the accepted control limits.
CSTE - Skill Category 1 - 46 © Copyright 2014 / All rights reserved
Process Control
Out of control processes
CSTE - Skill Category 1 - 47 © Copyright 2014 / All rights reserved
Common Causes of Variation
• All processes contain some inherent variation, or common causes of variation. The amount of variation in a process is quantified with summary statistics.
• In a computer operation, abnormal terminations cause variation. Typical common causes of abnormal terminations included invalid data, no disk space, errors in operating or job control instructions, etc.
CSTE - Skill Category 1 - 49 © Copyright 2014 / All rights reserved
Special Causes of Variation
• Special causes of variation are not present in a process
• They occur because of special or unique circumstances
• In the IT example of abnormal terminations in a computer operation, special causes might include:
– citywide power outages
– earthquakes or hurricanes
CSTE - Skill Category 1 - 51 © Copyright 2014 / All rights reserved
How Are Processes Brought Under Control?
• Understand process maturity
• Continued process improvement
• Testers should be a contributor
Do Testers Need to KnowStatistical Process Control (SPC)?
• The concept of measuring and reducing variability is commonly called statistical process control (SPC).
• Testers need to understand process variability, because the more variance in the process the greater the need for software testing.
CSTE - Skill Category 1 - 52 © Copyright 2014 / All rights reserved
List three things that could cause common cause of variation within your test processes and three things that could be special causes of variation impacting your test processes.
For this exercise, work in your small groups and be prepared to present your essay answer.
Essay (5 minutes):
CSTE - Skill Category 1 - 54 © Copyright 2014 / All rights reserved
• Producer view
–A deviation from specification: missing, wrong, or extra
• Customer view
–Anything that causes customer dissatisfaction, whether in the specifications or not
What is a Product Defect?
CSTE - Skill Category 1 - 55 © Copyright 2014 / All rights reserved
Why Are Defects Hard to Find?
• Not looking Tests often are not performed becausea particular test condition was unknown
• Looking but not seeingLike losing your keys onlyto discover they were in plain sight the entire time
• The size and complexity of applications often makes it impossible to test everything
CSTE - Skill Category 1 - 56 © Copyright 2014 / All rights reserved
Test Your SensitivityFirst read the sentence enclosed in the box that next appears
Then, count the F’s in the sentence.Count them only once.
Do not go back and count them again!
FINISHED FILES ARE THE RESULT OF YEARS OF SCIENTIFIC STUDY
COMBINED WITH THE EXPERIENCE OF MANY YEARS.
CSTE - Skill Category 1 - 57 © Copyright 2014 / All rights reserved
What Were Your Results?• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
• Number of F’s __________
CSTE - Skill Category 1 - 59 © Copyright 2014 / All rights reserved
Defects Typically Found in Software Systems AreThe Results of:
• IT improperly interprets requirements• Users specify the wrong requirements• Requirements are incorrectly recorded• Design specifications are incorrect• Program specifications are incorrect• Errors in program coding• Data entry errors• Testing errors• Mistakes in error correction• The corrected condition causes another defect
CSTE - Skill Category 1 - 61 © Copyright 2014 / All rights reserved
Process and Testing StandardsSome development processes are more effective than others. The need for standards within the Software Engineering discipline were noted. Three are listed.
• CMMI-Dev – A process improvement model for software development
• TMMI – A process improvement model for software testing
• ISO29119 – A set of standards for software testing
CSTE - Skill Category 1 - 62 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
CSTE - Skill Category 1 - 63 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
Culture 1: Manage People• Ad hock and chaotic
processes• Success is not repeatable• Political environment• If you are breathing, you
are at level 1!
CSTE - Skill Category 1 - 64 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
Culture 2: Manage Processes• Managed processes• Results predefined• Skills taught
CSTE - Skill Category 1 - 65 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
Culture 3: Defined• Processes rigorously defined• Tailored processes• Proactive process management
CSTE - Skill Category 1 - 66 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
Culture 4: Quantitatively Managed• Customer based objectives• Performance is predictable• Statistical analysis used
CSTE - Skill Category 1 - 67 © Copyright 2014 / All rights reserved
SEI’s Five Levels of Process Maturity
Culture 5: Optimization• Managed innovation• Continuous improvement• Process variations addressed
CSTE - Skill Category 1 - 68 © Copyright 2014 / All rights reserved
TMMI’s Five Levels of Test Maturity
http://www.tmmi.org/?q=node/64
CSTE - Skill Category 1 - 69 © Copyright 2014 / All rights reserved
You are currently operating at CMMI level 2 in your software testing department. Describe what it means to be operating at a level 2 maturity. Be specific and provide examples.
Essay (5 minutes):
CSTE - Skill Category 1 - 71 © Copyright 2014 / All rights reserved
The maturity of the productequals
the maturity of the process
CSTE - Skill Category 1 - 72 © Copyright 2014 / All rights reserved
ISO29119 – A set of process standards for software testing
There are 5 standards:
• ISO 29119-1: Concepts and Definitions
• ISO 29119-2: Test Processes
• ISO 29119-3: Test Documentation
• ISO 29119-4: Test Techniques
• ISO 29119-5: Keyword Driven Testing
CSTE - Skill Category 1 - 73 © Copyright 2014 / All rights reserved
Software TestingThe process of evaluating a deliverable with the intent of finding errors.
• Testing is not a stage/phase of the project • Testing is not just finding broken code• Testing is not a final exam • Testing is not “debugging”
What it’s Not!
CSTE - Skill Category 1 - 74 © Copyright 2014 / All rights reserved
General Testing Guidelines• Testing shows presence of defects
• Exhaustive testing is impossible
• Early testing
• Defect clustering
• Pesticide paradox
• Testing is context dependent
• Absence-of-errors fallacy
• Testing must be traceable to the requirements
• Defect data should be used to focus future efforts
• Testing should be done incrementally
• Focus on exceptions
CSTE - Skill Category 1 - 75 © Copyright 2014 / All rights reserved
Why Do We Test Software• To find defects• To reduce risk inherent in computer systems• To satisfy the customer• To prove that a program is good or no good• Developers are unable to build defect-free software• The user / customer does not know what they want• The development process is defective• To detect variations from specification/expectation• Establishing confidence that a program does what it is
supposed to do
CSTE - Skill Category 1 - 76 © Copyright 2014 / All rights reserved
Factors Affecting Software Testing
• People relationships
• Scope of testing ( producer vs. customer)
• Understanding life cycle testing
• Plan your work and work your plan
• Testing constraints
CSTE - Skill Category 1 - 77 © Copyright 2014 / All rights reserved
Factors Affecting Software Testing: People Relationships
Some attitudes causing a negative view of testing
• Testers hold up implementation, FALSE
• Giving testers less time to test will reduce the chance that they will find defects, FALSE
• Letting the testers find problems is an appropriate way to debug, FALSE
• Defects found in production are the fault of the testers, FALSE; and
• Testers do not need training; only programmers need training, FALSE!
CSTE - Skill Category 1 - 78 © Copyright 2014 / All rights reserved
Factors Affecting Software Testing: The Scope of Software Testing
• Software testing can help ferret out the true needs of the customer.
• Finding defects early in the software development process.
• Removing defects of all types prior to software going into production.
• Identifying weaknesses in the software development process.
CSTE - Skill Category 1 - 79 © Copyright 2014 / All rights reserved
Value of Life Cycle Testing
The Traditional view was that testing was a phase at the end of the project. This increased the cost because:
• errors found late in the product may cause severe consequences
• errors usually occur early in the development cycle.
CSTE - Skill Category 1 - 80 © Copyright 2014 / All rights reserved
Factors Affecting Software Testing: Testing Constraints
Movement of one of the testing constraints will
cause one or more of the other constraints to move
Scope Schedule
QualityResources Technology
CSTE - Skill Category 1 - 81 © Copyright 2014 / All rights reserved
Testing Cost Curve
After the optimum test point the cost of testing to uncover defects exceeds the losses from those defects.
CSTE - Skill Category 1 - 82 © Copyright 2014 / All rights reserved
Factors Affecting Software Testing: Misunderstanding Life Cycle Testing
Life Cycle Phase• Requirements• Design• Program (build/construction)• Test• Installation• Maintenance
CSTE - Skill Category 1 - 83 © Copyright 2014 / All rights reserved
Independent Testing• Ideally independent testers will report to the project manager
and have a reporting structure independent from the group designing or developing the application in order to assure that the quality of the application is given as much consideration as the project budget and timeline.
• The test manager’s responsibilities include:– Test planning and estimation– Designing the test strategy– Reviewing analysis and design artifacts– Chairing the Test Readiness Review– Managing the test effort– Overseeing acceptance tests
CSTE - Skill Category 1 - 84 © Copyright 2014 / All rights reserved
Independent Testing
• Testers are usually responsible for:– Developing test cases and procedures– Test data planning, capture, and conditioning– Reviewing analysis and design artifacts– Testing execution– Utilizing automated test tools for regression testing – Preparing test documentation– Defect tracking and reporting
CSTE - Skill Category 1 - 85 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
There are many Software Development Life Cycle Models• Ad-Hoc• Waterfall• V-Model• Incremental Mode• Iterative Development Model• Prototype/RAD Model• Spiral Model• Reuse Model
CSTE - Skill Category 1 - 86 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
All the models embrace the following in some format:• System Conceptualization• System Design• Unit Development• Software Integration and Testing• Site installation• Training• Implementation• Maintenance
CSTE - Skill Category 1 - 87 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Ad-hoc• System Capability is unpredictable• Performance depends on individuals• Repeatability depends on individuals
CSTE - Skill Category 1 - 88 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
WaterfallOne of the original models• System Conceptualization is completed• System Design is completed• Coding is completed• Testing is completedProblems• Real projects are uncertain and rarely follow the
sequential model• Process can be long
CSTE - Skill Category 1 - 89 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Waterfall
CSTE - Skill Category 1 - 90 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
V-Model• Extension of Waterfall• Shows the relationship between specification
development and the associated dynamic testing phase• Shows inverse relationship high level specifications
moving to details and testing more details to high level
CSTE - Skill Category 1 - 91 © Copyright 2014 / All rights reserved
A Life Cycle Quality Approach
Testing throughout development life-cycle
Early development of test requirements
Early detection of errors
At pre-determined points
Involves static and/or dynamic test techniques
The V-Model focuses on:
CSTE - Skill Category 1 - 92 © Copyright 2014 / All rights reserved
The “V” Testing ConceptOperational orBusinessNeed
VerifyOperational
or Business Need
DefineRequirements
VerifyRequirements
DesignSystem
VerifyDesign
BuildSystem
VerifyConstruction
AcceptanceTest
ValidateOperational
or Business Need
SystemTest
ValidateRequirements
IntegrationTest
ValidateDesign
UnitTest
ValidateConstruction
Static Dynamic
Validates
Validates
Validates
Validates
CSTE - Skill Category 1 - 93 © Copyright 2014 / All rights reserved
• User Acceptance Test
• System Test
• Integration Test
• Unit Test
Basic Test Stages
Construction &Unit Test
IntegrationTest
SystemTest
UserAcceptance Test
CSTE - Skill Category 1 - 94 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
IncrementalSuperset of Waterfall• Break Requirements into smaller buildable projects• Use full Lifecycle model for each smaller subset• Each successive increment adds functionality until the
final product is produced• Variants include Agile, Spiral, and RAD
CSTE - Skill Category 1 - 95 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Incremental
CSTE - Skill Category 1 - 96 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Iterative• Break Project into small parts• Get user feedback• Some variations allow immediate implementation into
productionProblems• Extensive user involvement• Communications take center stage• Requests for changes may cause confusion
CSTE - Skill Category 1 - 97 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Prototyping
• Creation of the major user interfaces without any substantive coding in the background in order to give the users a “feel” for what the system will look like
• Development of an abbreviated version of the system that performs a limited subset of functions; development of a paper system (depicting proposed screens, reports, relationships etc.)
• Use of an existing system or system components to demonstrate some functions that will be included in the developed system
CSTE - Skill Category 1 - 98 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Rapid Application Development (RAD)
Variation of Prototyping
• Less emphasis on detailed requirements
• User interacts with development team
• Four phases:Requirements Planning PhaseUser Design PhaseConstruction PhaseCutover Phase
CSTE - Skill Category 1 - 99 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Rapid Application Development (RAD)
CSTE - Skill Category 1 - 100 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Spiral Model
Waterfall plus Prototyping
• Initial version developed and then repeatedly modified
• Each version designed based on Waterfall method
• Spiral out from the centre
CSTE - Skill Category 1 - 101 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Spiral
CSTE - Skill Category 1 - 102 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Reuse Model
Build using Existing Components
• Library of Procedural Modules
• Library of Database Modules
• Borrow and use in the new function
CSTE - Skill Category 1 - 103 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Agile
Approaches which include the following are collectively known as Agile
• Scrum
• Crystal Clear
• Adaptive Software Development
• Feature Driven Development
• Dynamic Systems Development Methodology (DSDM)
• Extreme Programming
CSTE - Skill Category 1 - 104 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Agile Practices
• Detailed Planning for each iteration
• Test Driven Development
• Refactor Relentlessly
• Continuous Integration
• Paired Programming
• Onsite Customer
CSTE - Skill Category 1 - 105 © Copyright 2014 / All rights reserved
Software Development Life Cycle Models
Creating and Combining Models
Models should be combined or modified based the relationship between the Information System and its organizational environment. Environments can be classified
• Unchanging
• Turbulent
• Uncertain
• Adaptive
CSTE - Skill Category 1 - 107 © Copyright 2014 / All rights reserved
You feel that a Software Development Life Cycle testing model should be implemented in your testing approach. What justification would you provide to your manager to sell a defined life cycle approach?
Essay (5 minutes)
CSTE - Skill Category 1 - 109 © Copyright 2014 / All rights reserved
Testing Throughout the Lifecycle
• Continuous testing throughout the development process
• Must have formalized lifecycle
• Pre-determined deliverables
CSTE - Skill Category 1 - 110 © Copyright 2014 / All rights reserved
There are a number of constraints that must be recognized and planned for accordingly. An example of a constraint might be budgetary constraints. List 3 additional constraints of which a test manager might need to be aware and describe how trade-off decisions might be necessary regarding these three constraints.
Essay (10 minutes)
CSTE - Skill Category 1 - 112 © Copyright 2014 / All rights reserved
Static vs Dynamic TestingVerification – Static Test
• Walkthroughs
• Reviews
• Inspections
Validation – Dynamic Test• Test cases
• Scripting
• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing
• White Box
• Black Box
CSTE - Skill Category 1 - 113 © Copyright 2014 / All rights reserved
Verification and Validation Techniques
• Techniques for quantifiably assuring that a work product meets its stated objectives:
– Verification: Performed during development on key artifacts
• Walkthroughs, reviews and inspections
• Mentor feedback, training, checklists and standards
– Validation: Performed after a work product is produced
• Validating against established criteria
• Ensuring product integrates correctly into the environment
• Reactive approach focused on defect detection and removal
CSTE - Skill Category 1 - 114 © Copyright 2014 / All rights reserved
Verification and Validation Techniques
• Techniques for quantifiably assuring that a work product meets its stated objectives:
– Verification: Performed during development on key artifacts
• Walkthroughs, reviews and inspections
• Mentor feedback, training, checklists and standards
– Validation: Performed after a work product is produced
• Validating against established criteria
• Ensuring product integrates correctly into the environment
• Reactive approach focused on defect detection and removal
EXAM HINTThere is some variation in the industry regarding these definitions. Know them as stated for exam!
CSTE - Skill Category 1 - 115 © Copyright 2014 / All rights reserved
Walkthroughs, Reviews, Inspections
• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing
• Walkthroughs and inspections are very disciplined procedures aimed at removing the major responsibility for verification from the developer.
• The main goal of a review is to identify defects within the stage or phase of the project where they originate, rather than in later test stages; this is referred to as “stage containment.”
CSTE - Skill Category 1 - 116 © Copyright 2014 / All rights reserved
Walkthroughs, Reviews, Inspections
• Feasibility reviews• Requirements reviews• Design reviews• Code walkthroughs• Code Inspections• Requirements tracing
• Walkthroughs and inspections are very disciplined procedures aimed at removing the major responsibility for verification from the developer.
• The main goal of a review is to identify defects within the stage or phase of the project where they originate, rather than in later test stages; this is referred to as “stage containment.”
Described in detail in STBOK Section 6
CSTE - Skill Category 1 - 117 © Copyright 2014 / All rights reserved
Unit Testing Testing individual programs or components to
validate that the logic works according to specification
Validates technical quality of the code
Conducted by the developer who created the component
Begins once component development is complete
CSTE - Skill Category 1 - 118 © Copyright 2014 / All rights reserved
Integration Testing Tests the integration of components that have been
successfully unit-tested
Validates the technical quality of the design
Validates the integration of the application and the environment
Conducted by development with support from the test team
Begins once the first components have passed unit test
CSTE - Skill Category 1 - 119 © Copyright 2014 / All rights reserved
System Testing Tests the entire assembled system
Validates delivery of functional and non-functional requirements
Validates interface to upstream and downstream applications
Conducted by independent test team
Begins when integration testing has been successfully completed
CSTE - Skill Category 1 - 120 © Copyright 2014 / All rights reserved
User Acceptance Test
Validates system is “fit for use”
Evaluates how system integrates with manual business processes
Conducted by end users, customers or designated representatives
Begins when system test has been successfully completed
CSTE - Skill Category 1 - 121 © Copyright 2014 / All rights reserved
Essay (6 minutes)In software testing there are at least four stages of testing. Listthe sequence in which those four stages of testing should occurfrom one, (which is the first), to four, (which is the last). Brieflyexplain the objective of each of these four stages of testing.
CSTE - Skill Category 1 - 123 © Copyright 2014 / All rights reserved
ExerciseUnit and Integration testing primarily focuses on which of the following?
a) Verification
b) Validation
c) Test planning
d) Both a & c
Tests that validate system requirements are often called.
a) Functional Tests
b) Structural Tests
CSTE - Skill Category 1 - 126 © Copyright 2014 / All rights reserved
ExerciseUnit and Integration testing primarily focuses on which of the following?
a) Verification
b) Validation
c) Test planning
d) Both a & c
Tests that validate system requirements are often called.
a) Functional Tests
b) Structural Tests
EXAM HINTFor multiple choice parts, read each question’s stem twice, then read ALL the responses
CSTE - Skill Category 1 - 127 © Copyright 2014 / All rights reserved
Traceability Matrix
• Translation from Stage to Stage
• Requirements to be traced throughout the Lifecycle
Requirement ID Func Rqmt 1.1
FuncRqmt1.2
Func Rqmt 1.3
Func Rqmt 1.x
Func Rqmt
x.x
Technical Rqmt 1.1
Technical Rqmt 1.2
Technical Rqmt 1.x
TestCases 3 2 3 1 2 1 1 1
1.1.1 x
1.1.2 x x
CSTE - Skill Category 1 - 128 © Copyright 2014 / All rights reserved
Testing Schools of Thought
A ‘School of Thought’ is a system of beliefs shared by a group. Some of the Schools of thought are:
• Analytical School – testing is rigorous and technical
• Factory School – reduction of testing tasks to basic routines or repetitive tasks
• Quality (Control) School – process and standards
• Context-driven School –focus on product and people
• Agile School – continuous delivery of incremental products
CSTE - Skill Category 1 - 129 © Copyright 2014 / All rights reserved
Testing Approaches
The approaches do not relate to a model or school:
• Requirements-based Testing – quality of Requirements
• Risk-based Testing – list, prioritize, and test based on the assessed risk
• Model-based Testing – create a (simplified) model and build testcases based on the model
• Exploratory Testing - used by professional testers based on their knowledge of the system
• Keyword-driven Testing - define keywords or actions for each function to drive the testing
CSTE - Skill Category 1 - 130 © Copyright 2014 / All rights reserved
Test Categories and Testing Techniques
• Structural
• Functional
• Non-Functional
Will be discussed on the following slides.
CSTE - Skill Category 1 - 131 © Copyright 2014 / All rights reserved
Test Categories• Structural Tests
Tests that validate system architecture is structural testing. Also considered white box testing because knowledge of the internal logic of the system is used to develop test cases.
• Functional Tests
Tests that validate system are called functional testing. This testing addresses the overall behavior of the program by testing transacting flows, input validation, and functional completeness but no knowledge of the internal logic system is used (black box).
• Non-Functional Tests (Software Quality Factors)
Tests that validate system characteristics, such as performance, stability, maintainability, usability, and security. Can be considered a user’s point of view.
CSTE - Skill Category 1 - 132 © Copyright 2014 / All rights reserved
Structural Test TechniquesStructural system testing is designed to verify that the developed system and programs work.
CSTE - Skill Category 1 - 133 © Copyright 2014 / All rights reserved
• Structural test technique• Testing based on knowledge of internal code structure and
logic - usually logic driven
White-Box Testing
Internal View
What is the system doing?How is it doing it?
Input
Output
CSTE - Skill Category 1 - 134 © Copyright 2014 / All rights reserved
White-Box Techniques• Statement coverage
• Decision Coverage
• Condition Coverage
• Decision/Condition Coverage
• Multiple Condition Coverage
CSTE - Skill Category 1 - 135 © Copyright 2014 / All rights reserved
Functional Test TechniquesFunctional system testing ensures that the system requirements and specifications are achieved.
Technique Description ExampleRequirements System performs as specified. * Prove system requirements
* Compliance to policies, regulations
Error Handling Errors can be prevented or detected and then corrected.
* Error introduced into test* Errors re-entered
Intersystem Data is correctly passed from system to system.
* Intersystem parameters changed* Intersystem documentation updated
Control Controls reduce system risk to an acceptable level.
* File reconciliation procedures work* Manual controls in place
Parallel Old system and new system are run and the results compared to detect unplanned differences.
* Old and new systems
CSTE - Skill Category 1 - 136 © Copyright 2014 / All rights reserved
Black-Box Testing• Functional test technique
• Testing based on external specifications without knowledge of how the system is constructed - usually data or business process driven
• Validates that each input produces the appropriate output
External View
What did the application do?What should it have done?
Input
Output
CSTE - Skill Category 1 - 137 © Copyright 2014 / All rights reserved
Black-Box TechniquesEquivalence partitioning
Test cases generated using a subset of data to represent a larger class of data
Example:
A business rule edits credit limits within a given range ($10,000 - $15,000) could have three equivalence classes:
Less than $10,000 (Invalid)Between $10,000 and $15,000 (Valid)Greater than $15,000 (Invalid)
CSTE - Skill Category 1 - 138 © Copyright 2014 / All rights reserved
Black-Box Techniques (continued)
Boundary analysis
Test cases designed using data from the limits of the input and output domain classes
Example (continued)
Testing the boundaries of a business rule that edits credit limits within a given range ($10,000 - $15,000) would test:
Low boundary +/- one ($9,999 and $10,001)On the boundary +/- one ($10,000 and $15,000)Upper boundary +/- one ($14,999 and $15,001)
CSTE - Skill Category 1 - 139 © Copyright 2014 / All rights reserved
Black-Box Techniques (continued)
Other types include:
• Decision Table Testing
• State Transition Testing
• All-pairs (pairwise) Testing
• Cause-Effect Graphing
CSTE - Skill Category 1 - 140 © Copyright 2014 / All rights reserved
Black-Box Techniques (continued)
Ad Hoc or Error Guessing
Test cases and data designed based upon experience and intuition of the tester
Example
Testing the date-driven activities performed on credit reports, the tester might test the dates below based upon experience that year-end, month-end, and other dates usually cause problems:
2/29/2006
CSTE - Skill Category 1 - 141 © Copyright 2014 / All rights reserved
Non-Functional Test Types
• Accessibility Testing
• Conversion Testing
• Maintainability Testing
• Reliability Testing
• Stability Testing
• Usability Testing
CSTE - Skill Category 1 - 142 © Copyright 2014 / All rights reserved
Incremental Testing• Top-Down - testing begins with module
highest in the hierarchy using stubs to simulate lower interfacing modules; modules are added in descending hierarchical order
• Bottom-Up - testing begins with module at bottom of hierarchy using drivers or test harnesses to simulate higher interfacing modules; modules are added in ascending hierarchical order
CSTE - Skill Category 1 - 143 © Copyright 2014 / All rights reserved
Thread Testing• Often used during the integration of units or components
• Demonstrates key functional capabilities by testing a string of units or components that accomplish a specific function in the application
• Can be performed simultaneously with incremental testing
CSTE - Skill Category 1 - 144 © Copyright 2014 / All rights reserved
Regression Testing
Regression testing is a decision to re-test something that has already been tested looking for inadvertently introduced defects. Regression Testing should be performed:
• New releases of packaged software
• Application is enhanced or changes are made
• Support software changes
• Either side of a system interface is changed
• Changes to configuration
• Whenever changes are made after a testing stage is completed
CSTE - Skill Category 1 - 145 © Copyright 2014 / All rights reserved
Exercise
A. InspectionsB. Black BoxC. Capture/PlaybackD. Stress TestingE. Regression Testing
1. Testing previously verified logic
2. Testing Branches/paths
3. Re-use of test data
4. Test correct implementation of specifications
5. Testing processing limits
Match the items to the numbered list
1
2
34
5
CSTE - Skill Category 1 - 147 © Copyright 2014 / All rights reserved
Skill Category 1 - SummarySoftware Testing Principles and Concepts
The following topics were discussed in this Skill Category:
• Vocabulary & Quality Basics
• Understanding Defects
• Process and Testing Published Standards
• Software Testing
• Software Development Life Cycle (SDLC) Models
• Agile Development Methodologies
• Testing Throughout the SDLC
• Testing Schools of Thought and Approaches
• Test Categories and Testing Techniques