driving quality throughout the application lifecycle

Richard ErwinDeveloper Tools SpecialistMicrosoft UK

27th June 2011

Driving Quality Throughout the Application Lifecycle

Define

Design

Develop

Test

Release

Maintain

Phases of the Application Lifecycle

Test

Develop

Cost of BugsCo

st /

£

Lifecycle stageDevelopment Testing UAT Released

Team Foundation Server

Application Lifecycle ToolingBusiness Analyst Eclipse Developer

Tech Writer on Mac OS X

Visual Studio Developer

Tester

C++ developer on Linux

• Dev Process Support• Version Control • Automated Build• Test Case

Management• Reporting

Code Quality ToolingUnit

Testing& TDD

Code Coverag

e

Test Impact

AnalysisPex

Static Code

AnalysisCode

Metrics

Code Profiling

Style Cop

Check-In Policies

Automated Build

Reporting Shelving

Test Tools Security Tools

Database Tools

Architecture Tools

Unit Testing Code Analysis

Other ToolsProcess

Free Pro Premium Ultimate TFS

Unit Testing

Unit Testing

− What is it?− Ensuring individual units of source code work

− A developer, not tester, activity− Unit test ≠ functional test

− Why do it?

Quality Flexibility Delivery Time

Test Driven Development

− Grown out of eXtreme Programming− Write Test before Implementation− Improve Code Coverage− Can help With Implementation Design− Catch Defects Early

Make it Fail• No code without a failing test

Make it Work• As simply as possible

Make it Better• Refactor

PEX

− Automatically generate test suites with high code coverage.

− Visual Studio add-in for testing .NET Framework applications

− Moles - Isolation framework for .NET

Code Coverage

− How much unit testing should you do?− How much of the code is being tested?

Unit TestingDEMO

Code Analysis

Code Metrics

− Is the Code Maintainable?− Where are Defects Likely to be Found?− Where Should I Concentrate my Efforts?

Code Metrics

Maintainability Index

Cyclomatic Complexity

Depth of Inheritance

Class Coupling

Lines of Executable

Code

Maintainability Index

0 – 9 Low Maintainability

10 – 19 Moderately Maintainable

20-100 Good Maintainability

MAX(0,(171 - 5.2 * ln(Halstead Volume) 0.23 * (Cyclomatic Complexity) – 16.2 * ln(Lines of Code))*100 / 171) 

Code Analysis

− My code:− Compiles− Passes Unit Testing− Isn’t Too Complex

− It must be OK then, right?

Code Analysis

Code Profiler

StyleCop

− Ensure Uniform Code Formatting− Improve Code Readbility− Aid Code Readability− Removes Debate Among Team Members− Apply At Check-In− Examples

− Braces {} on a New Line− Use Tabs instead of Spaces− Empty Line Separation Between Methods

Code AnalysisDEMO

Additional Tools & Processes

Check In Policies− Add quality gates to Version Control

− Create your own

Branch Visualizations

View Timeline

Track Changes

against WIs

Visualize Branch

Hierarchy

Branches are a first

class artefact

Automated Builds

• Automate a manual task• Eliminate dependencies on key personnel

Repeatability

• Find errors earlier• Minimise “bad builds”• Improve integration

Quality

• Quickly understand the impact of changes• Understand the “pulse” of the project• Scheduled -> Continuous Integration

Confidence

Did it Build?Did my Tests

Pass?Code Coverage?Code Analysis?

What Code Changed?

Which Features Were Impacted

Which Tests Were

Impacted?

What about Previous Builds?

Testing & Debugging Tools

• Microsoft Test Manager & Team Foundation Server

Test Case Management

• Automatic Data Collection• Fast Forward for Manual Testing

Manual Testing

• Automated Functional Testing

Coded UI Testing

Web Performance Testing

Load Testing

• Post Mortem Debugging

IntelliTrace

Dedicated tool for Testers. Integrated with TFS

Test Cases & Shared Steps are TFS Work

Items

Link Test Cases to other

Work Items (e.g.

requirements)

Define Test Settings - What data we want to

automatically collect when tests are executed

31

Generate TFS Bug Work

Items from Microsoft Test

Runner

32

Give the bug a title – the hard work has been automatically

done

Intellitrace

Every interesting

event is logged

Post mortem debug

34

Walk up and down the call

stack

Debug code executed by

someone else

Prototype with SketchFlow

Understand Existing

Code Assets

Drill in and follow

relationships

Enforce application architectur

e

Database Tools in Visual Studio*

Schema Comparison

Data Comparison

Refactoring

Data Generation

Database Unit Testing

Static Code Analysis

*Visual Studio 2010 Premium/Ultimate

Database Development Schema Comparison & SQL Unit Tests

Database DevelopmentGenerate

Realistic Test Data

Security ToolsMSF Agile + Secure Development Lifecycle Process Template

SDL Threat Modelling Tool

Anti-Cross Site Scripting (Anti-XSS) Libraray

Microsoft Code Analysis Tool .NET (CAT.NET)

BinScope Binary Analyzer

MiniFuzz File Fuzzer

SDL RegEx Fuzzer

Attack Surface Analyzer Beta

http://www.microsoft.com/security/sdl/adopt/tools.aspx

Define

Design

Develop

Test

Release

Maintain

Phases of the Application Lifecycle

Keep in touch

• UK Visual Studio events list:• http://www.microsoft.com/visualstudio/en-gb/visual-studio-even

ts

Future Events

• @ukvs• Feedback and suggestions• Competitions

Twitter

• ukvs@microsoft.com • Questions and queries including licensing

Email

• http://blogs.msdn.com/ukvsts

Blog