testing kurt schmidt intro testing - drexel cci · 2018-12-05 · testing kurt schmidt intro...
TRANSCRIPT
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing
Kurt Schmidt
Dept. of Computer Science, Drexel University
December 5, 2018
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
A QA engineer walks into a bar:A QA engineer runs into a bar:A QA engineer skips into a bar:
She orders 1 beerShe orders -1 beersHe orders 99,999 beersIt orders sfojbeerkiwHe orders ’0 ; select id, password from users ; dropbeers ; commit’
Opening day a customer walks in, asks for the bathroom.The bar catches fire, everybody dies.
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Intro
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing and Debugging
Testing “...a determined, systematic attempt to break aprogram that you think is working.”
Running a program with the intent offinding bugs“...testing can demonstrate the presenceof bugs, but not their absence.”
Debugging Finding the cause of an error, and fixing it.
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Goals of Testing
Discover bugs, not to show that program worksReduce the risk of failure to an acceptable levelDesigning a test before writing code is a great way toreduce bugs
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Complete Testing
Complete testing coverage is generally not possibleOr, not practical
E.g., consider a program that takes an input of 10characters
280 distinct inputsAt 1µs/test, would take more than twice the age of theuniverse
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing Caveats
People naturally assume what they do is correctPeople by nature overlook minor deficiencies in theirown workEasy to overlook or ignore bad resultsEasy to choose only test cases that show the program“works”It’s useful to get another’s help
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Software Testing Myths
Really good programmers don’t have bugsTesting implies an admission of failureTesting is a punishment for our errorsTesting can be avoided if we
ConcentrateUse OO methodsUse a good programming language
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Software Testing Reality
Humans make mistakesEspecially when creating complex artifacts
Even good programs have 1-3 bugs per 100 lines ofcodePeople who claim they write bug-free code likelyhaven’t coded much
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Defensive Programming
Anticipate potential problemsDesign, code the system so problems are accountedfor, or detected as early as possible
Defensive design – Minimise confusion due to complexityDefensive coding – Take steps to localise problems
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Defensive Design
Simplicity of designEncapsulationDesign with error in mindPrototype, walk-throughMake all assumptions and conditions explicitly
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Encapsulation
Minimise coupling (dependencies) between objects1
Provide a sufficient interfaceHide all data behind the interfaceHave enough functionality so that the client needn’taccess the data directly
1Loosely; i.e., a collection of related data, not necessarily an instanceof a class
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Designing With Error in Mind
Error handling is often added as an afterthoughtShould be part of the interface
Decided before/as you code
Ask “What if?” often
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Design Reviews
Show the design to another programmer, or leadDiscussions and critiques are an excellent way to learnIt is much easier to see another’s mistakes, andassumptions
Another pair of eyeballs is always helpful
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Pre- and Post-Conditions
Pre-conditions Assumptions that are made by a function ormethod upon entry, of necessity or efficiency
Units or state of argumentsState of the objectState of globals
Post-condition Anything a function guarantees upon exit (ifpre-conditions were met
Return valueAny side-effects
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Class Invariants
Conditions on an object which are always trueFor a Vector V of length n, elements V0 through Vn−1are validA Fraction might always be in lowest terms
Conditions public (interface) functions can assumeupon entryMust guarantee are true upon exit
Note, this does not necessarily apply to helper functions
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Evolutionary Programming
Compile and test your code oftenImplementation is bottom-up
Write small piecesTest themGather into bigger pieces
Evolve (and test) your program by writing modules,using stubs
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Testing
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
General Testing Guidelines
A necessary part of a test case is the expectedoutput/behaviorTest border conditionsTest error conditionsCheck that the program doesn’t do what it shouldn’t(harder to impossible)Never be the last to test your own codeTesting is an art; your skills will develop
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Functional vs. Structural Testing
Functional (black box) testing:Implementation details are invisibleCode is tested against its specs – look at advertisedbehavior
Inputs, outputsSide effectsError codes and exceptions
Cases can be written before the code is writtenStructural (clear box) testing:
Tester must be familiar with the codeExercise all control paths, loop boundariesTools such as gcov might be helpful
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Integration, Unit, System, and AcceptanceTesting
These labels often get in the way of talking abouttesting1
I don’t want to confuse this discussion this term“Black-box” does not imply “unit” testing“Clear-box” does not imply “integration” testing
1Mike Kelly, searchsoftwarequality.techtarget.com/
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
The assert statement
Used during development, testing and debugging totest programmer’s assumptions
Not to be used in released code
#include <assert.h>...assert( x!=0 ) ;
If condition failsPrint conditionPrint a line numberDump the program
To turn it off:
$ gcc -DNDEBUG ...
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Debugging
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Types of Bugs
SyntacticDesignLogic
InterfaceMemory
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
How to Find a Problem
ThinkI’ll spend a few minutes with print/debug statementsI’ll move to a debugger before too longIf you reach an impasse, sleep on itIf you reach an impasse, show it to someone else
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Why Use a Debugger?
Debuggers are very powerfulDesk checking code can be tedious and error-pronePrint/Debug statements might require re-compilation
Error-proneLeaves a mess to clean up
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Basic Common Debugger Functions
BreakpointsSet at line or functionCan be conditional
Run programExecute line at a timeRun to next breakpointMove execution pointer aroundSet command-line argumentsSee stack trace
DataEvaluate any variable, expressionModify values in memory
Testing
Kurt Schmidt
Intro
TestingAssertions
Debugging
Think Before Repairing Errors
Be careful you fix the problemDon’t simply address the symptom
Remember, don’t fix bad code, rewrite it!