6656075 software testing chapter10
TRANSCRIPT
Test Cases
Writing & Tracking Test Cases
What is a test case?
A set of inputs, execution conditions and expected results developed for a particular objective.
The smallest entity that is always executed as a unit from beginning to end.
Goals of Test Case Planning?
Carefully & Methodically planning test cases is important.
Why?Organization.Repeatability.Tracking.Proof of testing / not testing.
Organization: Even in small project possible to have
thousands of tests cases. Test cases may be created by several
testers over a long span. Proper planning will organize test cases
so that all testers and other project members can review & use them effectively.
Goals of Test Case Planning?
Repeatability Necessary to run the same tests
cases several times to look for new bugs.
Without proper planning it would be impossible to know what test cases were last run and how they were run.
Keeping info on this will help to repeat the exact tests.
Goals of Test Case Planning?
Tracking We may to answer several questions
like: How many test cases planned for run? How many test cases were run in the
last release? How many passed & how many failed? Any tests cases skipped? Planning would help to answer these
questions easily.
Goals of Test Case Planning?
Test Case Planning Overview
TEST PLAN
Test Design Spec
Test DesignSpec
Test CaseSpec
Test ProcedureSpec
Increasing Emphasis onProcess of Planning
Increasing Emphasis onWritten Plan
Test Design Spec
Take the high level overall test plan.Break it into specific features and testable items.Assign the items to individual testers.Only general mention of automation, black-box or white-box.No details.Where to get next level of detail??
Go to test design specification.
ANSI/IEEE 829:Refines the test approach defined in the test plan.Organizes and describes the testing that needs to be performed on a specific feature.Identifies: the features to be covered by the
design and its associated tests. the test cases & test procedures
required to accomplish testing and mentions the pass/fail criteria.
What is Test Design Spec?
Identifiers: A unique identifier used to reference
and locate the test design spec. The spec should reference the overall
test plan. Should contain pointers to any other
plans or specs that it references.
Parts of Test Design Spec
Features to be Tested. Description of s/w feature covered by
the test design. Ex. “The addition function of Calculator”.
Also identify features that may be indirectly tested as a side effect of testing the primary feature. (UI for load/stress)
Also list features being covered in this plan but wont be tested.
Parts of Test Design Spec
Approach Description of the general approach that will
be used to test the features. Expand on the approach to describe the
technique to be used and explain how the results will be verified.
Ex. A testing tool will be developed to sequentially load and save pre-built data files of various sizes. The # of data files, sizes and the data they contain will be determined through black-box tests and supplemented with white-box. A pass or fail will determined by comparing using the file compare tool.
Parts of Test Design Spec
Test Case Identification
A high level description and references to the specific tests cases that will be used to check feature.
List selected equivalence partitions and provide references to the test cases and test procedures used to run them.
Ex. Check the highest possible valueTest Case ID# 15326
Parts of Test Design Spec
Pass/Fail Criteria
Describe exactly what constitutes a pass and a fail of the tested feature.
Must be simple and clear. No doubt on what constitutes a pass
and a fail. Can be fuzzy. (10% or more test
cases fail)
Parts of Test Design Spec
What is Test Case Specification?
How to document the test cases created. ANSI/IEEE 829.A test case specification documents the actual values used for input along with the anticipated outputs.A test case also identifies any constraints on the test procedure resulting from use of the specific test case.
Parts of Test Case Specification
Identifiers A unique identifier for the test case spec to be
referenced by the test design spec and test procedure.
Test item Describes the detailed feature, code module
and others being tested. More detailed description than test design
spec. Ex. Test Design Spec: “The addition function of
Calculator”. Test Case Spec: “Upper limit handling of
addition calculation”.
Input Specification: Lists all I/Ps or conditions given to the
s/w to execute the test case. Ex. To a calculator addition function it
would be 1+1
Output Specification: Describe the result expected from
executing the test case. Ex. Did 1+1 =2?
Parts of Test Case Specification
Environmental Needs: H/w, s/w, test tools, facilities, staff etc
needed to run the test case.
Special Procedural Requirements: Describes anything unusual that must
be done to perform the test. Ex. Testing nuclear power plant need
special requirements.
Parts of Test Case Specification
Test case dependencies: Is a test case dependent on any other
test case? Document that info here.
Parts of Test Case Specification
Test Case Presentation
Can be presented as: Simple Lists. Outlines. Graphical Diagrams: state table/data
flow diagrams Matrix/table form.
Test Case Id: Unique identification number. Simple scheme T-number.
Test Outline: The item in the test case spec from
which the test case derived. Helps to track traceability.
Test Case Presentation: Table Format
Prior State: Idle state Successful execution of another test.
Input: The input sequence to be entered by the
tester.
Expected Results: Behaviour the tester expects to observe.
Actual Results: To be recorded after execution.
Test Case Presentation: Table Format
Test Case Presentation: Table Format
TestCase ID
TestOutline
PriorState
Input ExpectedResults
ActualResults
T-1 1.1.1.1 Idle 199 Display:Error
TestCase ID
PrinterMessage
Model Mode Options
T-1 1.1.1.1 Idle 199 Display:Error
A sample test caseIdentificationTest case ID: _____ S/W Version: ______Subsystem: username field test OS: _____Tester’s Name: _____ Date of Test: ____________________________________________________Initial Setup1. Bring up the login dialog box.2. …._______________________________________________Input:1. Place the cursor in the username field2. 2….____________________________________________________Expected Results:……….__________________________________________________Actual Results: Pass Fail
Test ProceduresAfter test design and test cases we need procedures required to execute the test cases.ANSI/IEEE 829.Test procedure specification identifies all steps required to operate the system and exercise the specified test cases in order to implement the associated test design. Also called test script spec.
Parts of Test Procedure Spec
Identifier: Unique identifier to tie the test procedure with
the associated test design and test case.
Purpose: The purpose of the procedure and reference to
the test cases that it will execute.
Special Requirements: Other procedures, special testing
skills/equipment etc needed.
Procedure Steps: Detailed description of how the tests are to run.
Log: Tells how and by what methods the results and
observations will be recorded.
Setup: Explain how to prepare for the test.
Start: Explain how to prepare for the start of the test.
Procedure: Describes the steps used to run the tests.
Measure: Describe how the results are to be determined:
with stop watch/visual examination/…..
Parts of Test Procedure Spec
Shut Down: Explain steps for suspending the test for
unexpected reasons.
Restart: How to pick up the test at a certain point if there
is a failure.
Stop: Describes the steps for an orderly halt.
Wrap Up: How to restore the environment to its pre-test
condition.
Contingencies: Explains what to do if things don’t go as planned.
Parts of Test Procedure Spec
Example of a test procedure
Identifier: WinCalcProc98.1872Purpose: This procedure describes the steps necessary to
executeAddition function test cases WinCalc98.051 through
WinCalc98.0185Special Requirements: No special h/w or s/w is required
other thanwhat is outlined.Procedure Steps:
Log: The tester will use WordPad with the Testlog template as the means for taking notes while…..Setup: The tester must install a clean copy of Win98….Start:Procedure:
Organizing & Tracking Test Case
In your head: if for own use.Paper/Documents.Spreadsheet.Custom d/b.
Test Case Management ToolsWhen there is a need to manage several disorganized tests.The best test case managers are: Provide a user interface for managing tests. Organize tests for ease of use and maintenance. Start and manage test execution sessions that run
user-selected tests. Provide seamless integration with capture/playback
and coverage analysis tools. Provide automated test reporting and documentation.
Though several leading vendors claim to accomplish this the bad news is we are still not there.
Write a test design spec, test case spec and test procedure spec.
Ex. Performing the four basic mathematics operations +, -, *, / on the calculator.