unit testing
Post on 01-Nov-2014
10 Views
Preview:
DESCRIPTION
TRANSCRIPT
Unit testing in Visual Studio 2010
Because it is a matter of quality!!!
Different Type of Tests
Unit Tests
Manual Tests
Web Tests
Coded UI Tests
Automated tests aimed to verify the functionality of a little piece of code, run very often by developers and from automated build.
Test Suites managed with Microsoft Test Manager, manually executed from testers and associated with User Stories. Can be automated with Action RecordingWeb Request record and replay to simulate the interaction with web sites. Can be used to do load or stress test to verify the behavior of software under heavy usageRecord and replay user interaction with UI, plugin-based recorder to maximize reproducibility, generation of UiMap that contains all the data used to reproduce the automation.
Test and Lifecycle - Waterfall
Requirements
Design Develop DeployTest
In waterfall processes testing has a well-defined phase, located between Develop and Deploy. With such a process developers tend to do manual test to verify that requirements are satisfied
Modern waterfall approaches tend to distribute testing in all the phases of the process.
Test and Lifecycle - Waterfall
Test
Requirement are software artifacts that can be tested following basic principle of testingThe purpose of testing a program is to find problems
in it
The purpose of finding problem is to get them fixed(Testing Computer Software – Cem Kaner et Al.)Requirements
Requirements should be verified with stakeholders and users to verify that we are going to create the right software
A mismatch between the program and its specification is an error in the program if and only if the specification exists and is
correct
A program that follows a terrible specification is terrible not perfect
(Testing Computer Software – Cem Kaner et Al.)
Test and Lifecycle - Waterfall
Test
Even with Emerging Architecture you need to validate the design of the software.The design phase should produce a «proof of concept» for each «typical» part of the softwareEarly testing on POC should be used to validate the infrastructureTechnique used: Unit Test, Performance Test
Design
Testing of the Design phase should take place when: No existing architecture, start from scratch First project with new technology Strict performance requirements External service Verify testability of the architecture
Test and Lifecycle - Waterfall
Test
During developement all form of testing are usedUnit test: used by developers to validate new codeManual test: Used by test team to validate code for «done» requirementsCoded UI test: Used by developers and test team to automate testing through the interface
Develop
The main purpose of testing during development is guarantee quality of the codebase.When code is out of the developing phase, it should be ready to be “beta tested” to find bug that are not caught during development.
Test and Lifecycle - WaterfallTesting phase is where you find bug during real-usage of the softwareThe software is usually in «beta» stage, meaning that it can be used by real user.This is the moment where you mainly use Acceptance test with Microsoft Test Manager
Develop
All defect found during this phase should be filled up in issue/bug tracker to be investigated by the teamNo developer should be allowed to «pick a bug» to fix it without a formal review.The aim of this phase is to fix all most important bugs of the software, as well as gather user impression of the overall product, to make little modification before the real shipping.All steps should be reported.
Real-usage Formal bugfix
Quality
Test and Lifecycle - Agile
Each agile process has its own development cycle, but quite all of them have an interactive cycleEach interaction value is added to the software and this value should be immediately available to StakeholdersAcceptance and Unit Testing are the skeleton of this lifecycle, without tests it is difficult to be really Agile. Testing permits to
Establish Definition Of Done (Acceptance) Refactor and do incremental development (Unit testing) Clarify User Stories (Acceptance) Assure internal quality of the codebase (Unit testing in
CI)
Requirements
Design Develop DeployTest
Unit Testing
xUnit test pattern and typical techniques of unit testing in
VS2010
Unit test principleAutomated: Test can be executed without user interaction
Develop
Integrated: Written in the same language of the code to test, available in the same environment
Standard: Many frameworks available
DevelopAgile: Support for refactoring and agile lifecycle
DevelopPattern support: www.xunitpattern.org estabilish a ground language and series of pattern for unit testing in all language/environments
xUnit framework pattern
Setup
Setup: Initial conditions are set to maximize reproducibility of the test
Exercise: Code to be tested is executed, usually is a call to a single function or a series of calls.
Verify: An expectation is checked, if the expectation fails, the whole test is considered to be failed
Teardown: If the test modify the system/environment everything is restored to a good / clean state
Exercise
Verify
Teardown
Quality of good testsAutomated: Test can be executed without user interaction
Develop
Indipendent: The test does not depend on any other test
Focused: The test should verify a single condition
DevelopRepetable: Test depends only on precondition contained in the setup phase and it should give the same result at each executionDevelopFast: Slow tests are not executed frequently
Test is First-Class code
Part of the project
Part of the project: testing code has the same importance of tested code
Refactored: Time is spent refactoring and adapting test code to the modifications of tested code.
Time: writing good tests takes time but having good Unit Test suite dramatically raise quality
Long run ROI: Sadly enough, Unit Testing usually does not pay in the short time, so it is difficult to take the path to unit testing
Refactored
Hard to write
Plan for the future
DEMO
Basic of 4-phase test in Visual Studio 2010
Test smells
High maintenance
Symptom: too much time to maintain testsImpact: test are not executed and removed from suiteCause: Code duplication, low quality test code
Symptom: test fails but the code under test is correctImpact: time wasted to look for inexistent bugsCause: Test is too complex
Symptom: test has conditional logic insideImpact: it is not clear what it is testingCause: Dependence from environment, wrong logic
Symptom: it is difficult to understand what is testedImpact: it is difficult to understand why it failedCause: Eager test, mystery guest, irrelevant information, hard coded test data
Buggy tests
Conditional test logic
Obscure test
Avoid test smellsUsers does not execute the test often
Develop
Continuous integration automatically executes tests
Test are slow, they needs several minutes to be executed
DevelopCategorized test permits execution of only a little part of a test, continuous test tools (Mighty Moose, Test Impact) can automate thisDevelopManual intervention before test run
DevelopWrite unit test first, write test that is testable
Fixture
The most difficult part to create Unit Tests without smells is fixture
management
The fixture
Setup Exercise
Verify Teardown
Everything is needed to create preconditions for the execution of test, it can be data in database, setting global variables, cleaning up file on disks, etc.
This is the most complex part of the whole Unit-Testing stuff, because it is the source of many smell
Fresh Fixture – transient: The fixture is recreated at each test, at the end of the test the system is restored to the original stateFresh Fixture – persistent: Same as transient, but the system is not restored at the end of the testShared Fixture: The fixture is shared for many test, it is created, then a series of test is executed
Fresh transientFixture is created during the setup phase, take track of every modification that is done to the system.It is useful to create a mechanism that permits to schedule automatic cleanup at the end of the test.
Setup
Exercise
Verify
TeardownFixture is removed during the teardown phase, everything is restored. Garbage collector should not be used because it is not-deterministic.If multiple restore operations should be executed, use try-catch clause to be sure to cleanup the most of them
Fresh persistentFixture is created during the setup phase, no need to track operationsSetup
Exercise
Verify
Teardown
Setup
Exercise
Verify
Teardown
Next test will create another fixure, but every modification done by the previous test is still thereThis can augment the risk of interacting-test smell.This technique is useful only if we are sure that the fresh persistent fixture will not impact other tests.It is useful mainly if the restore operation is slow and you do not want to lose time.
Shared fixtureFixture is created during the setup phase, modification are trackedSetup
Exercise
Verify SetupExercise
Verify
Teardown
After the latest verification the fixture is removed.
SetupExercise
Verify
Teardown
SetupExercise
Verify
Subsequent tests execution will use the same fixture
First test is executed, the test is composed only by Exercise and Verify steps
Shared fixtureFixture Setup Setup
Exercise
Verify
Teardown
Setup
Exercise
Verify
Teardown
Setup
Exercise
Verify
Teardown Teardown
A shared fixture is created before the execution of each testEach test can create another specific fixture and remove it in Setup/TeardownAfter the latest test of the suite is executed the final Teardown is run to remove the shared fixture.
Assertion
How to write good verification code that maximize the
advantage of unit testing
AssertionSingle assertion: A test should verify only one fact of the code under test
Develop
Clear assertion: Reading verification code should immediately clarify what fact we are testing
Test named assertion: Name of the test should clarify the purpose and what is verified by the test.
Writing good assertionsToo many assertions in the test
Develop
Expected Object, create an object that reflect the status you want to check, name that object in clear way.
Complex assertion code, that is not readable.
DevelopUse FluentInterface based assertion frameworks (SharpTestEx)
DevelopMagic values to verify return values or object properties
DevelopUse good names for variables and expected values
Test doubles
Creates fake component to be sure to exercise only a single part
of the system
Depend On ComponentThe System Under Test dialogates with external component and the verification phase has not access to it.
Setup
Exercise
Teardown
VerifySUT DOC
If the test fails, we are not sure if the reason is in bug in the SUT or for a bug or different behavior of the DOC.It leads to erratic test and frequent debugging test smell.
Test Double
We need to design the SUT to loose coupling with the DOC, with this technique you can create a double of the DOC in the test.The Test Double simulates the real component, but it has a well defined and reproducible behavior that is defined in the Setup phase
Setup
Exercise
Teardown
VerifySUT
Test Double
Verifiable Test Double
With a verifiable Test Double the Verify step can ask to Test Double to verify how the SUT interacted with him during Exercise phase.As an example, if you create a Test Double of an E-Mail sender component, you can verify that during Exercise the SUT asked to send an E-Mail.
Setup
Exercise
Teardown
VerifySUT
Test Double
Types of TestDoubles
Stub
Stub: its only purpose is to answer to the requests of the SUT with well defined and known answers, or with a default one
Dummy: It does nothing, simply avoid the SUT to crash for missing components.
Mock and Spy: Behave like a Stub, but also permits to make assertion on how the SUT interacted with them.
Fake Object: It implements a DOC with minimal functionality.
Dummy
Mock Object and Test Spy
Fake Object
Database Testing
Testing a database is complex and needs specific techniques
Database Testing Basic
Database Sandbox
Sandbox: Each tester/environment should have a dedicated database to execute testing. Minimize dependency, maximize execution time avoid conflicts
Database Fixture: Shared fixture to minimize setup time, use of Back Door Manipulation and backup/restore attach/detach method. Heavy use of Shared Fixture.Smart teardown: Use preload at each test and make every test transactional so you can rollback after each verification phase
Database Projects: Specific project type introduced with VS2008 that contains a dedicated section to unit testing.
Fixture Setup
Teardown
Database Project
……
top related