12. testing and debugging - testing, bug fixing and debugging the code
Post on 22-May-2015
1.205 Views
Preview:
DESCRIPTION
TRANSCRIPT
Testing and Debugging
Testing, Bug Fixing and Debugging the Code
Yordan Dimitrov
Telerik Corporationwww.telerik.com
Team Leader, TeamPulse,
Telerik Corporation, Blog:http://blogs.telerik.com/
jordandimitrov/
http://codecourse.telerik.com/
Testing and Debugging
Testing, Bug Fixing and Debugging the Code
Yordan DimitrovTelerik Corporationwww.telerik.
com
Table of Contents What Is Testing? Seven Testing Principles Developer Testing
Developer vs. QA Testing
Debugging vs. Testing
Black-box vs. White-box Testing
Role of Developer Testing in Software Quality
Recommended Approach to Developer Testing
3
What Is Testing?
Main Test Activities Testing is not just running tests, but also: Planning and control Choosing test conditions Designing and executing test cases Checking results Evaluating exit criteria Reporting on the testing process
and system under test Finalizing or completing closure
activities6
Main Objectives in Testing
Testing pursues several objectives: Finding defects Gaining confidence about the level
of quality Providing information for decision-
making Preventing defects
7
Different Viewpoints / Different Objectives
Objectives of testing differ according to the point of view: Development testing – cause as
many failures as possible and fix them
Acceptance testing – confirm that the system works as expected
Assessment – assess the quality of the software and its readiness for release 8
Different Viewpoints / Different Objectives (2)
Objectives of testing differ according to the point of view: Maintenance testing – check for
new defects, introduced during development
Operational testing – assess system characteristics such as reliability or availability
9
Seven Testing Principles
Seven Testing Principles (1)
1.Testing shows presence of defects Testing can show that defects are
present Cannot prove that there are no
defects Appropriate testing reduces the
probability for defects
11
Seven Testing Principles (2)
2.Exhaustive testing is impossible All combinations of inputs and
preconditions are usually almost infinite number
Testing everything is not feasible Except for trivial cases
Risk analysis and priorities should be used to focus testing efforts
12
Seven Testing Principles (3)
3.Early testing Testing activities shall be started as
early as possible And shall be focused on defined
objectives
The later a bug is found – the more it costs!
13
Seven Testing Principles (4)
4.Defect clustering Testing effort shall be focused
proportionally To the expected and later observed
defect density of modules
A small number of modules usually contains most of the defects discovered Responsible for most of
the operational failures
14
Seven Testing Principles (5)
5.Pesticide paradox Same tests repeated over and over
again tend to loose their effectiveness Previously undetected defects
remain undiscovered
New and modified test cases should be developed
15
Seven Testing Principles (6)
6.Testing is context dependent Testing is done differently in
different contexts Example: safety-critical software is
tested differently from an e-commerce site
16
Seven Testing Principles (7)
7.Absence-of-errors fallacy Finding and fixing defects itself
does not help in these cases: The system built is unusable
Does not fulfill the users’ needs and expectations
17
Developer TestingTesting as a Priority of the Developer
Developer vs. QA Testing
Software is tested in numerous ways Some are typically performed by
developers
Some are more commonly performed by specialized test personnel QA Engineers
19
Developer Testing Developer testing refers to testing by the developer
Usually the following tests are priority of developer testing: Unit tests
Component tests
Integration tests
Sometimes regression tests and system tests are also included
20
Testing as a Priority of the QA Engineer
Numerous additional kinds of testing are performed by QA Engineers and rarely performed by developers:
21
Beta tests
Customer-acceptance tests
Performance tests
Configuration
tests
Platform tests
Stress tests
Usability tests
Etc.
Debugging vs. Testing Testing
A means of initial detection of errors
Debugging A means of diagnosing and
correcting the root causes of errors that have already been detected
22
Black-box vs. White-box Testing
Testing is usually broken into two broad categories: Black-box testing
White-box testing
24
Black-box Techniques Black-box techniques are a way to derive and select test conditions, test cases, or test data Based on an analysis of the test
basis documentation Also called specification-based or
behavioral techniques Tests are based on the way the
system is supposed to work
25
White-box Techniques White-box techniques
Also called structural or glass-box techniques
Based on an analysis of the structure of the component or system Information about how the software
is constructed?
E.g., code and detailed design information
Usually a priority of developers26
Role of Developer Testing in Software
Quality
Role of Developer Testing in Software
Quality Individual testing steps typically find less than 50 percent of the errors present each (Unit test, component test, and
integration test) The combination of testing steps often finds less than 60 percent of the errors present (Jones 1998)
28
Why is Testing a Hard Activity?
Testing's goal runs counter to the goals of other development activities
Testing can never completely prove the absence of errors
Testing by itself does not improve software quality
Testing requires you to assume that you'll find errors in your code
29
How Much Testing is Enough?
How much testing should be done is a matter of risk: Too much testing can delay the
product release and increase the product price
Insufficient testing hides risks of errors in the final product
30
How Much Time Should Be Spent in Developer
Testing ?
31
Developer testing should probably take 8 to 25 percent of the total project time
Recommended Approach to
Developer TestingGround Rules and Tips for Effective Development and Testing
Recommended Approach to Developer
Testing Test for each relevant requirement
Make sure that the requirements have been implemented
Test for each relevant design concern Make sure that the design has been
implemented
33
Recommended Approach to Developer
Testing (2) Use "basis testing" to add detailed test cases to those that test the requirements and the design
Use a checklist of the kinds of errors you've made on the project to date or have made on previous projects
34
Test First or Test Last?
Effort is the same
Detect defects earlier
Forces you to think at least a little bit
Exposes requirements problems sooner
Run it when you want
35Source: flickr
Limitations of Developer Testing
Developer tests tend to be "clean tests“
Developer testing tends to have an optimistic view of test coverage
Developer testing tends to skip more sophisticated kinds of test coverage
36
Bag of Testing Tricks Incomplete Testing Structured Basis Testing Data-Flow Testing Equivalence Partitioning Error Guessing Boundary Analysis Classes of Bad Data Classes of Good Data Use Test Cases That Make Hand-
Checks Convenient37
Structured Basis Testing
Test each statement in a program at least once
Compute the minimum number of test cases: Start with 1 for the straight path
through the routine
Add 1 for each of the following keywords, or their equivalents: if, while, repeat, for, and, and or
Add 1 for each case in a case statement If the case statement doesn't have a
default case, add 1 more
38
Sample
Statement1; <-- 1Statement2;
if ( x < 10 ) { <-- 2 Statement3;}
Statement4;
(1)Count "1" for the routine itself.(2)Count "2" for the if.
39
Sample Test Cases
40
Case
Minimum memory use Most readable output
1 Nominal case All boolean conditions are true
2 The initial for condition is false numEmployees < 1
3The first if is false
m_employee[ id ].governmentRetirementWith-held >=MAX_GOVT_RETIREMENT
4 The second if is false because the first part of the and is false
not m_employee[ id ].WantsRetirement
5 The second if is false because the second part of the and is false
not EligibleForRetirement( m_employee[id] )
6The third if is false
not EligibleForPersonalRetirement( m_employee[ id ] )
Data-Flow Testing
The normal combination of data states A variable is defined, used one or
more times, and perhaps killed
41Source: http://blog.radvision.com
Data-Flow Testing (2)
The key to writing data-flow test cases is to exercise all possible defined-used paths: All definitions
Test every definition of every variable
I.e., every place at which any variable receives a value
All defined-used combinations Test every combination of defining a
variable in one place and using it in another 42
Sample Test Casesif ( Condition 1 ) {
x = a;} else {
x = b; } if ( Condition 2 ) {
y = x + 1; } else {
y = x - 1; }
43
Case Test Description
1 Condition 1 = true, Condition 2 = true
2 Condition 1 = false, Condition 2 = false
?x = a; y = x – 1;
?x = b; y = x + 1;
Sample Test Cases
44
Case Test Description
7 Define companyRetirement in line 12, and use it first in line 26
This isn't necessarily covered by any of the previous test cases
8 Define companyRetirement in line 12, and use it first in line 31
This isn't necessarily covered by any of the previous test cases
Boundary Analysis
45
Case Test Description
1 Case 1 is defined so that the true condition for m_employee[ ID ]. governmentRetirementWithheld < MAX_GOVT_RETIREMENT is the first case on the true side of the boundary
3 Case 3 is defined so that the false condition for m_employee[ ID ]. governmentRetirementWithheld < MAX_GOVT_RETIREMENT is on the false side of the boundary
9 An additional test case is added for the case directly on the boundary in which m_employee [ ID ].governmentRetirementWithheld = MAX_GOVT_RETIREMENT
Compound Boundaries
Minimum and Maximum allowable values
46
Case Test Description
10 A large group of employees, each of whom has a large salary (what constitutes "large" depends on the specific system being developed)—for the sake of example, we'll say 1000 employees, each with a salary of $250,000, none of whom have had any social security tax withheld and all of whom want retirement withholding.
11 A group of 10 employees, each of whom has a salary of $0.00.
Classes of Bad Data
Too little data (Case 2-11)
Too much data The wrong kind of
data The wrong size of
data Uninitialized data
47
Case Test Description
12 An array of 100,000,000 employees. Tests for too much data.
13 A negative salary. Wrong kind of data.
14 A negative number of employees. Wrong kind of data.
Classes of Good Data Nominal cases—middle-of-the-road,
expected values Minimum normal configuration Maximum normal configuration Compatibility with old data
48
Case Test Description
16 A group of one employee. To test the minimum normal configuration.
17 A group of 500 employees. To test the maximum normal configuration.
Typical Errors Which Classes Contain the Most Errors?
Errors by Classification The scope of most errors is fairly
limited Many errors are outside the domain
of construction Most construction errors are the
programmers' fault Clerical errors (typos) are a
surprisingly common source of problems
49
Typical Errors (2) Errors by Classification
Misunderstanding the design is a recurring theme in studies of programmer errors
Most errors are easy to fix It's a good idea to measure your
own organization's experiences with errors
50Source: http://kathrynvercillo.com
Improving Your Testing
Planning to Test Retesting (Regression Testing) Automated Testing
51
Finding a Defect1.Stabilize the error
2.Locate the source of the errora)Gather the data
b)Analyze the data and form hypothesis
c)Determine how to prove or disprove the hypothesis
d)Prove or disprove the hypothesis by 2c
3.Fix the defect
4.Test the fix
5.Look for similar errors52
Finding a DefectDemo
Tips for Finding Defects (1)
Use all available data Refine the test cases Check unit tests Use available tools Reproduce the error several different ways
Generate more data to generate more hypotheses
Use the results of negative tests Brainstorm for possible hypotheses54
Tips for Finding Defects (2)
Narrow the suspicious region of the code
Be suspicious of classes and routines that have had defects before
Check code that’s changed recently Expand the suspicious region of the code
Integrate incrementally Check for common defects Talk to someone else about the problem
Take a break from the problem
55
Fixing a Defect Understand the problem before you fix it
Understand the program, not just the problem
Confirm the defect diagnosis Relax Save the original source code Fix the problem not the symptom Make one change at a time Add a unit test that expose the defect
Look for similar defects
56Source: http://www.movingseniorsbc.com
Your ego tells you that your code is good and doesn't have a defect even when you've seen that it has one.
How "Psychological Set" Contributes to Debugging Blindness
Psychological Considerations
57
Psychological Considerations (2)
58
Psychological Distance
How "Psychological Distance" can help
59
First Variable
Second Variable
Psychological Distance
stoppt stcppt Almost invisibleshiftrn shiftrm Almost nonedcount bcount Smallclaims1 claims2 Smallproduct sum Large
Testing and Quality Testing can give confidence in the quality of the software if it finds few or no defects
If defects are found, the quality increases when those defects are fixed
Lessons learnt from previous mistakes improve future performance
62
форум програмиране, форум уеб дизайнкурсове и уроци по програмиране, уеб дизайн – безплатно
програмиране за деца – безплатни курсове и уроцибезплатен SEO курс - оптимизация за търсачки
уроци по уеб дизайн, HTML, CSS, JavaScript, Photoshop
уроци по програмиране и уеб дизайн за ученициASP.NET MVC курс – HTML, SQL, C#, .NET, ASP.NET MVC
безплатен курс "Разработка на софтуер в cloud среда"
BG Coder - онлайн състезателна система - online judge
курсове и уроци по програмиране, книги – безплатно от Наков
безплатен курс "Качествен програмен код"
алго академия – състезателно програмиране, състезания
ASP.NET курс - уеб програмиране, бази данни, C#, .NET, ASP.NETкурсове и уроци по програмиране – Телерик академия
курс мобилни приложения с iPhone, Android, WP7, PhoneGap
free C# book, безплатна книга C#, книга Java, книга C#Дончо Минков - сайт за програмиранеНиколай Костов - блог за програмиранеC# курс, програмиране, безплатно
?
? ? ??
?? ?
?
?
?
??
?
?
? ?
Questions?
?
Testing and Debugging
http://academy.telerik.com
Free Trainings @ Telerik Academy
“High-Quality Programming Code" course @ Telerik Academy codecourse.telerik.com
Telerik Software Academy academy.telerik.com
Telerik Academy @ Facebook facebook.com/TelerikAcademy
Telerik Software Academy Forums forums.academy.telerik.com
top related