09_functional_testing.ppt
TRANSCRIPT
COMP 6710 Course NotesSlide 9-1Auburn UniversityComputer Science and Software Engineering
Course Notes Set 9:
Functional Testing
Computer Science and Software EngineeringAuburn University
COMP 6710 Course NotesSlide 9-2Auburn UniversityComputer Science and Software Engineering
• Dynamic (running) black-box (blindfolded) testing
• Also known as behavioral testing
• Based on specification- if one not available, the software is the spec
• Exam inputs/outputs: needs test cases
Functional Testing
COMP 6710 Course NotesSlide 9-3Auburn UniversityComputer Science and Software Engineering
• Test-to-pass
- make sure the software minimally works- don’t push it to the limit- apply simplest and/or straightforward cases- not to find bugs, initially- do this test FIRST
Purpose of Testing
COMP 6710 Course NotesSlide 9-4Auburn UniversityComputer Science and Software Engineering
• Test-to-fail
- after test-to-pass- design and run test cases with the purpose to break the software- probe the known and unknown weaknesses- errors forcing
Purpose of Testing
COMP 6710 Course NotesSlide 9-5Auburn UniversityComputer Science and Software Engineering
Functional (Black Box) Testing
• Knowing the specified function (requirements), design test cases to ensure that those requirements are met.– Example : Sort (list);
• Structural Testing - How well is the code exercised?• Functional Testing - How well does Sort perform its intended
function?
• In general, complete functional testing is not feasible– Attempting to test every possible input to the function
• A randomly selected set of test cases is statistically insignificant– “Not all test cases are created equally”
• Test case selection– Based on characteristics of input and output sets relative
to specified functionality.
COMP 6710 Course NotesSlide 9-6Auburn UniversityComputer Science and Software Engineering
Functional Testing
• Types of errors looked for during functional testing– Incorrect function or missing function– Interface errors– External database errors– Performance errors (including stress testing)– Initialization/termination errors
• Tests are designed to answer the following questions– How is functional validity tested?– What classes of input will make good test cases?– Is the system particularly sensitive to certain input values?– How are the boundaries of a data class isolated?– What data rates and data volume can the system tolerate?– What effect will specific combinations of data have on system
operations?
[Adapted from Software Engineering 4th Ed, by Pressman, McGraw-Hill, 1997]
COMP 6710 Course NotesSlide 9-7Auburn UniversityComputer Science and Software Engineering
Goals and Methods of Functional Testing
• Goals– Produce test cases that reduce the overall number of
test cases– Generate test cases that will tell us something about
the presence or absence of errors for an entire class of input
• Methods/Approaches– Equivalence partitioning– Boundary value analysis– Matrix of functional possibilities– Cause-effect graphing– Decision Tables
COMP 6710 Course NotesSlide 9-8Auburn UniversityComputer Science and Software Engineering
Equivalence Partitioning
• It is impossible to test all cases• Equivalence partitioning provides a systematic means
for selecting cases that matter and ignoring those that don’t
• An equivalence class or equivalence partition is a set of test cases that tests the same aspect or reveals the same bugs
e.g., If X >= 15 then do-this else do-that
(- 15) 15 (15 )
COMP 6710 Course NotesSlide 9-9Auburn UniversityComputer Science and Software Engineering
Equivalence Partitioning
• Equivalence partitions – groups for similar inputs, outputs, and/or operation of the software
e.g., file-name, 1 .. 255 characters- valid characters- invalid characters- valid length- invalid length
COMP 6710 Course NotesSlide 9-10Auburn UniversityComputer Science and Software Engineering
Equivalence Partitioning
e.g., copy operation- copy menu- c or C- Ctrl-c or Ctrl-Shift-c
• Fully tested in the first effort, equivalence partitioning (1 case each) test for new versions
• Goal: to reduce the set of possible test cases
• Too few partitions => may not reveal all catchable bugs
COMP 6710 Course NotesSlide 9-11Auburn UniversityComputer Science and Software Engineering
Equivalence Partitioning
• Equivalence partitioning divides the input domain of a program into classes of data from which test cases can be derived– Ideally, each test case could uncover classes of errors, thereby
reducing the total number of test cases that must be developed• Input condition - some kind of condition placed on the input
– Typically a specific value, a range of values, a set of related values, or a Boolean condition
• Equivalence Class - a set of valid or invalid states for input conditions– Range - 1 valid and 2 invalid equivalence classes are defined– Specific Value - 1 valid and 2 invalid equivalence classes are
defined– Set - 1 valid and 1 invalid equivalence class are defined– Boolean - 1 valid and 1 invalid equivalence class are defined
[Adapted from Software Engineering 4th Ed, by Pressman, McGraw-Hill, 1997]
COMP 6710 Course NotesSlide 9-12Auburn UniversityComputer Science and Software Engineering
Example
Ϭ¹¹¹¹¹¹¹¹¹Þßàfunction in_list (input1:name_type;ϧÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏinput_list : list_names)ϧÏreturn boolean isϪ˹¹¹¹¹¹¹¹ÏϧÏíÏp : list_names;ÏϧbeginÏϨ¹¹Ïp := input_list;ÏϨ¹¹±while not(p = null) loopÏϧÏÏ·¹³´if Ada.Strings.Fixed.Index ÏϧÏϯϵ§(Source => String(p.name),ÏϧÏϯϵ§Pattern => String(input1)) /= 0 then ÏÂÄÏϯϵ¾¹¹Ïreturn true;ÏϧÏϯ϶´else ÏϧÏϯϸ¾¹¹Ïp := p.next_name;ÏϧÏϯÏÈÏend if;ÏϧÏÏ°end loop;ÏÂĹ¹Ïreturn false;
Equivalence Classes:(1) Inputs where input1 is in the list
(2) Inputs where input1 is not in the list
Specific Input Partitions:List input1Empty ?One element In the listOne element Not in the list>One element First element>One element Last element>One element Middle element>One element Not in list
Test CasesList input1 Output<nil> ? falsebird bird truebird fish falsebird, cat, owl bird truedog, pig, chicken chicken true
...
COMP 6710 Course NotesSlide 9-13Auburn UniversityComputer Science and Software Engineering
Boundary Value Analysis
• Range : a..b
– Example : 100..200• Test cases : 99, 100, 101, 199, 200, 201
• Number of values– Test cases that exercise minimums and maximums
• Apply the above to the output conditions– Try to drive output to invalid range
• Internal data structures with boundaries– Example : A(1..100) with test cases A(0), A(1), A(2), A(99),
A(100), A(101)– A(0) and A(101) should generate exceptions
a b
( ))( (
COMP 6710 Course NotesSlide 9-14Auburn UniversityComputer Science and Software Engineering
Boundary Condition Test Cases
• If software can operate on the edge of its capabilities, it will almost certainly operate well under normal conditions
• For I = 1 to 10 data (I) = -1;
end;
10 elements, data(0), data(1), data(2) data(9), data(10), data(11)
COMP 6710 Course NotesSlide 9-15Auburn UniversityComputer Science and Software Engineering
Boundary Condition Test Cases
• Boundary conditions for a legitimate triangle
• Boundary conditions for side classification
• Boundary conditions for angle classification
• Valid input/extremes
COMP 6710 Course NotesSlide 9-16Auburn UniversityComputer Science and Software Engineering
Boundary Condition Test Cases
• Types of Boundary conditionsnumeric character position quantityspeed location size
Also, extremes
first/last min/max start/finish over/underempty/full Shortest/longest slowest/fastest
largest/smallest …
COMP 6710 Course NotesSlide 9-17Auburn UniversityComputer Science and Software Engineering
Boundary Condition Test Cases
• Partitions- boundary- one or two valid points inside the boundary- one or two invalid points outside the boundary
e.g., First – 1 / Last + 1 Smallest –1 / Largest + 1
COMP 6710 Course NotesSlide 9-18Auburn UniversityComputer Science and Software Engineering
Sub-boundary Conditions
• Also known as Internal boundaries
• Bit, nibble, byte, word, K, M, G, T
• Why? E.g., 256 commands, 15 are frequently used. Needs only a nibble.
0XXXX nibble, 1XXXXXXXX byte
COMP 6710 Course NotesSlide 9-19Auburn UniversityComputer Science and Software Engineering
Sub-boundary Conditions
• ASCCI table – boundaries not obvious
• Default, empty, blank, null, zero, none (may be of a separate equivalence partition and treated individually)
• Invalid, wrong, incorrect, garbage data(test to fail)
COMP 6710 Course NotesSlide 9-20Auburn UniversityComputer Science and Software Engineering
Matrix of Functional Possibilities
• Input/Output Conditions– If the number of combinations of input/output is manageable, then
consider using a matrix of functional possibilities– Especially useful if the input/output combinations are enumerated
in the requirements specification
• Example : Input (or output) will be a combination of {A,B} and {x,y,z}
x y z
A
B
COMP 6710 Course NotesSlide 9-21Auburn UniversityComputer Science and Software Engineering
Example : The Triangle Problem
• Input– 3 floating point numbers
• Processing– Determine if the 3 numbers form a triangle
• If not, print message “Not a Triangle”• If it is a triangle
– Classify according to side : equilateral, isosceles, scalene– Classify according to largest angle : acute, right, obtuse
• Output– List the 3 numbers– List the classification or “Not a triangle”
COMP 6710 Course NotesSlide 9-22Auburn UniversityComputer Science and Software Engineering
MFP for the Triangle Problem
Acute Obtuse Right
Scalene
Isosceles
Equilateral
Additional Functional Test Cases (if any):
COMP 6710 Course NotesSlide 9-23Auburn UniversityComputer Science and Software Engineering
Cause Effect Graphing
• Causes (input conditions) and effects (actions) are listed for a module, and an identifier is assigned to each
• A cause-effect graph is developed– Looking for causes without effects– Looking for effects without causes
• The graph is converted to a decision table (if a decision table has been used as a design tool, developing the graph and table is not necessary)
• Decision table rules are converted to test cases
COMP 6710 Course NotesSlide 9-24Auburn UniversityComputer Science and Software Engineering
Cause-Effect Graph Symbology
Identity
“Not”
“And”
“Or”
Symbology Constraints
c1
c1
c1
c2
c1
c2
e1
e1
e1
e1
c
a
a
a
a a
b
b
b
b b
E I O
R M
ExclusiveInclusive
Only One
Requires Masks
COMP 6710 Course NotesSlide 9-25Auburn UniversityComputer Science and Software Engineering
Cause-Effect Graphing Example
• The CHANGE subcommand - used to modify a character string in the “current line” of the file being edited– Inputs
• Syntax : C /string1/string2• String1 represents the character string you wish to
replace– 1-30 characters– Any character except ‘/’
• String2 represents the character string that is to replace string1
– 0-30 characters– Any character except ‘/’
• At least one blank must follow the command name “C”
COMP 6710 Course NotesSlide 9-26Auburn UniversityComputer Science and Software Engineering
Cause-Effect Graphing Example
– Outputs• Changed line is printed to the terminal if the
command is successful• “NOT FOUND” is printed if string1 cannot be
found• “INVALID SYNTAX” is printed if the command
syntax is incorrect
– System Transformations• If the syntax is valid and string1 can be found in
the current line, then string1 is removed and string2 replaces it
• If the syntax is invalid or string1 cannot be found, the line is not changed
COMP 6710 Course NotesSlide 9-27Auburn UniversityComputer Science and Software Engineering
Cause-Effect Graphing Example• Cause 1: The first nonblank character following the “C” and
one or more blanks is a ‘/’• Cause 2: The command contains exactly two ‘/’ characters• Cause 3: String1 has length 1• Cause 4: String1 has length 30• Cause 5: String1 has length 2-29• Cause 6: String2 has length 0• Cause 7: String2 has length 30• Cause 8: String2 has length 1-29• Cause 9: The current line contains an occurrence of string1• Effect 1: The changed line is typed• Effect 2: The first occurrence of string1 in the current line is
replaced by string2• Effect 3: NOT FOUND is printed• Effect 4: INVALID SYNTAX is printed
COMP 6710 Course NotesSlide 9-28Auburn UniversityComputer Science and Software Engineering
Complete Cause-Effect Graphc1
c2
c3
c4
c5
c6
c8
c7
c9
i1
i2
i3
e1
e2
e3
e4
COMP 6710 Course NotesSlide 9-29Auburn UniversityComputer Science and Software Engineering
Converting to a Decision Table
1 2 3 4 5 6 7 8 9 10 11
c1c2
c3c4c5
c6c7c8
c9
e1e2e3e4
I = invoked A = absentS = suppressed P = presentX = don't care