database unit testing team edition for database professionals sachin rekhi program manager microsoft...
TRANSCRIPT
Database Unit TestingTeam Edition for Database Professionals
Sachin RekhiProgram ManagerMicrosoft Corporationhttp://blogs.msdn.com/sachinre
Agenda
• Team Data Overview• Data Generation• Database Unit Testing
● Overview● Designer Experience● Test Validation● Test Execution● Exploring the Generated Code● Data-driven Testing● Custom Verification Logic● Transaction Support● Negative Testing● Custom Test Conditions● Integration with Application Unit Tests
3
Incorporate the Database Professional into the software lifecycle and Incorporate the Database Professional into the software lifecycle and provide them with a foundation for change management and process provide them with a foundation for change management and process
integration.integration.
Incorporate the Database Professional into the software lifecycle and Incorporate the Database Professional into the software lifecycle and provide them with a foundation for change management and process provide them with a foundation for change management and process
integration.integration.• Change Management
● Project Based Development• Project Model that represents schema as objects providing a “personal sandbox” for offline
development that lives within a Visual Studio Solution
• Team Collaboration with Work Item and Process Integration with Team Foundation Server
● Automated Change Support• Rename Refactoring with the ability to preview pending changes prior to execution
• Comparison Tools (Schema & Data Compare) allow comparisons & synchronization of schema and data with design/test/production databases
• Version Control of all database objects with the ability to reverse engineer a database to bring it under Source Control
● Database Unit Testing• Database Unit Testing ensures iterative changes don’t break existing functionality
• Data Generator provides Repetitive Dataset Generation for tests based on saved settings
• Build / Deployment● MSBuild Integration for Database Deployments/Builds based on Projects ● Either Create a new Database at the target location or Update an Existing Schema
4
Visual Studio Team SystemVisual Studio Team SuiteVisual Studio Team Suite
MS
F P
rocess a
nd
Gu
idan
ce
MS
F P
rocess a
nd
Gu
idan
ce
Visual Studio Team Foundation ServerVisual Studio Team Foundation Server
VisualVisualStudioStudio
IndustryIndustryPartnersPartners
SoftwareSoftwareArchitectsArchitects
SoftwareSoftwareDevelopersDevelopers
SoftwareSoftwareTestersTesters
DatabaseDatabaseProfessionalsProfessionalsVisualVisual
StudioStudioTeamTeam
ExplorerExplorerApplication Application ModelingModeling
Infrastructure Infrastructure and Deployment and Deployment
ModelingModeling
Code AnalysisCode Analysis
Performance Performance TuningTuning
Security AnalysisSecurity Analysis Database Database Build &Build &
Deployment Deployment
Database Database Change Change
ManagementManagementDatabase Database TestingTesting
Performance Performance TestingTesting
Manual TestingManual Testing
Test Case Test Case ManagementManagement
Visual Studio Professional EditionVisual Studio Professional Edition
Change ManagementChange Management
Work Item TrackingWork Item Tracking
ReportingReporting
Project SiteProject Site
Integration ServicesIntegration Services
Project ManagementProject Management
Load Test AgentLoad Test Agent
Visio and UML Visio and UML ModelingModeling
Class ModelingClass Modeling
Unit TestingUnit Testing
Code Code CoverageCoverage
New!New!
6
To create a solid foundation for testing we support data generation
Deterministic – always generate the same layout
Matched to your schema and very customizable
Easily update generation plan to reflect schema changes
Bottom Line: Realistic values with representative distributions
Data GenerationOverview
Data GenerationDesign Time
• Setting up Data Generation implies defining:● Which generator to use● Which distribution to attach to the generator● Changing setting on the generator & distribution● The numbers of rows to generate● Optionally defining ratios between related tables
• By default:● Each column is bound to the generator matching the column data
type• FK columns are mapped to the Foreign Key generator
• Uniqueness is inferred from PK, UC constraints and indexes
• Using the Uniform distribution when not unique
Data GenerationDesign Time
Value generators• Simple generators for each data type
● Strings: ASCII and Unicode ((var)char, n(var)char, (n)text)● Numbers: tinyint, smallint, int, bigint, real, float, decimal, numeric,
money● Binary ((var)binary, image)● Date and Time● UniqueIdentifier (GUID)● Bit
• Complex generators● Foreign Key, Regular Expression, Data Bound
• Distributions● Uniform, Normal, Inverse Normal, Exponential, Inverse
Exponential
Data GenerationDistributions @ Work
Data Generation Distributions
0
2000
4000
6000
8000
10000
12000
1 501 1001 1501 2001 2501 3001 3501 4001 4501
# of Rows
Ge
ne
rate
Va
lue
(0
-10
00
0)
Uniform
Normal
InverseNormal
Exponential
InverseExponential
Data GenerationDesign Time
• Understand domain constraints● Check constraints (min/max)
• Table cardinality● Enforce table ratios● Ex) Orders \ OrderDetails
Data GenerationExecuting a Data Generation Plan
• Validation of● Security requirements
• Fails when security requirements are not met!
● Target schema against DGEN definitions• Fails the generation when bindings do not match!
• Optionally purge tables● Required to guarantee repeatable data generation
• Spin up parallel streams of INSERT statements● Based on relationships between tables● Number of connections used is currently gated by the schema
relationships
• Configurable Error Thresholds
Data GenerationCustomization & Extensibility
• Customization of value generation● Regular Expression Generator● Data Bound Generator
• Extensibility● Custom Generator● Custom Distribution
• Ensures that database changes do not break existing Ensures that database changes do not break existing codecode
• Generate automated tests for programmability objectsGenerate automated tests for programmability objects• Develop a battery of tests to run with every database Develop a battery of tests to run with every database
buildbuild• Design & customize your tests in the language of your Design & customize your tests in the language of your
choicechoice● T-SQL, VB.NET\C#T-SQL, VB.NET\C#
• Builds on existing Team Test unit testing functionalityBuilds on existing Team Test unit testing functionality• Integrate your database tests along side your application Integrate your database tests along side your application
unit testsunit tests
Database Unit TestingOverview
• Develop your database unit test through an easy to use database test designer
• Test Script Generation● Automatically generate test script stub for
• Stored Procedures, Functions, Triggers
• Test Script● SQL to exercise the object under test
• Pre\Post Test Scripts● SQL executed prior to and after test script to setup & cleanup
test
• Test Initialize\Cleanup Scripts● Common SQL executed prior to and after every test
Database Unit TestingDesigner Experience
• Two primary methods of validating database unit tests● T-SQL Assertions
• RAISERROR
● Test Conditions• Easily configured UI-based client side assertions• Supported types
– Empty ResultSet– Not Empty ResultSet– Row Count– Scalar Value– Execution Time– Inconclusive
Database Unit TestingTest Validation
• Automatic Deployment Integration● Automatically deploy database project prior to running
tests
• Data Generation Integration● Automatically generate data based on generation plan
prior to running tests
• Execution & Validation connection strings● Validation connection can be higher privileged
account
Database Unit TestingTest Execution
• Ways to execute database tests● From Visual Studio
• Test Menu \ Test View \ Test Manager \ Test Results• F5 Experience
● Command-line• Mstest.exe
● Team Build• Execute tests post Team Build
Database Unit TestingTest Execution
• Database unit test designer round-trips generated VB.NET\C# code
• ADO.NET code to● Execute SQL against target● Validate test conditions
• Exposed code enables variety of customization scenarios
Database Unit TestingExploring the Generated Code
• Have the ability to parameterize inputs in test script
• Feed inputs from a database table• Easily test a variety of inputs in a single test
case• Requires customization of VB.NET\C# code
Database Unit TestingData-driven Testing
• Add custom verification logic in VB.NET\C#• Exposed ExecutionResult object
● Exposes resultset data returned from test script● Exposes execution time & rows affected info
• Take advantage of Classic Team Test Assertions
Database Unit TestingCustom Verification Logic
• Data Generation● Used to ensure consistent database state
prior to test run
• Transaction Support● Ensures consistent state during test run● Two Methods
• Transactions in T-SQL• System.Transactions in VB.NET\C#
Database Unit TestingTransaction Support
• Expected error cases are as important to test as main success scenarios
• 3 methods● SQL – Try\Catch● VB.NET\C# - Try\Catch● VB.NET\C# - Expected Exception Attribute
Database Unit TestingNegative Testing
• Create re-usable advanced verification logic • Fully supports test condition extensibility• Author a new test condition that derives from
TestCondition base class in any .NET language• Complete integrated UI experience
Database Unit TestingCustom Test Conditions
• Create VS solution containing both app & db projects
• Support for heterogeneous test lists• Run application & database tests side-by-side• Ensure database changes don’t break data
access through app tests
Database Unit TestingIntegration with Application Tests
• Database Unit Testing Database Unit Testing ensures quality in the data tierensures quality in the data tier• Data Generation to support repeatable data generation Data Generation to support repeatable data generation
for testingfor testing• First class unit testing framework for database First class unit testing framework for database
developersdevelopers• Integrate your database tests along side your application Integrate your database tests along side your application
teststests
Database Unit TestingWrap Up
28
Pricing, Licensing and Availability
• Included in Team Suite at No Extra Cost• Purchase as an individual Edition
● Same pricing as other Team System Editions• http://msdn.microsoft.com/vstudio/howtobuy/default.aspx
• Availability● CTP 7 Available Today● RTM by the end of 2006
29
Resources
• Product Website & CTP7 Download● http://msdn.microsoft.com/vstudio/teamsystem/products/dbpro/
default.aspx
• Product MSDN Forums● http://forums.microsoft.com/MSDN/ShowForum.aspx?
ForumID=725&SiteID=1
• Blogs● http://blogs.msdn.com/sachinre
• Wealth of blog posts, videos, and web casts on database unit testing tips, tricks, and best practices
● http://blogs.msdn.com/camerons● http://blogs.msdn.com/gertd