creating robust, maintainable coded ui tests with visual studio 2012 brian keller sr. technical...

Creating Robust, Maintainable Coded UI Tests with Visual Studio 2012 Brian KellerSr. Technical EvangelistMicrosoft Corporation

DEV312

I work here…

…on this…

…and this.

http://bit.ly/ThisWeekC9

I blog here:http://blogs.msdn.com/briankel

http://tinyurl.com/ALM2010Book http://tinyurl.com/TFSBook

http://tinyurl.com/ALM2012 http://tinyurl.com/ProTFS2012

Objectives•Find more UI regression bugs by employing coded UI tests.

•Build and maintain robust coded UI tests.

•Create coded UI tests which result in fewer false positives.

What You’ll Need

Visual Studio 2012

Premium or Ultimate

C# or VB skills

Microsoft Test Manager

(optionally)

Team Foundation Server

(optionally)

Testable application

Test Automation Platform SupportFully supported platform

Partial solution. Further work required in future to complete

Best efforts with known issues. Users can workaround these cases in the code. No major ongoing investment.

Currently no support but on the roadmap for future releases

Currently no support and none planned for now. Opportunity for partners to add this using the Extensibility support. Docs and samples around extensibility are here.

For latest info refer to this article.

Platform Support Notes

IE7/8/9– HTML/AJAX IE9 partially supported in SP1.

Windows Forms 2.0+.NET Fx Controls fully supported, working on 3rd party support.

WPF 3.5+.NET Fx Controls fully supported, working on 3rd party support.

SharePoint 2007 & 2010 See blog post for more info.

FireFox – HTML/AJAXSupported in Feature Pack 2 for FF 3.5 & 3.6.

SilverlightSupported in Feature Pack 2 for Silverlight 4, in-browser apps.

Windows Win32May work with some known issues, but not officially supported.

Dynamics (Ax)Partially supported – see article. Dynamics CRM web client is supported.

MFCPartial support – Known issues & Workarounds

Citrix/Terminal ServicesThe client (MTM or VS) need to be on the remote machine.

Office Client Apps

IE 6/Chrome/Opera/Safari Opportunity for partners to add this using the Extensibility support. Docs and samples around extensibility are here.Flash/Java

SAP

demo

Coded UI Tests

Get Used to Using Using. It’s Useful!

using (ApplicationUnderTest.Launch(this.UIMap.LaunchAppParams.UIMyWindowsFormsAppWindowExePath, this.UIMap.LaunchAppParams.UIMyWindowsFormsAppWindowAlternateExePath)){this.UIMap.SimpleTest();…}

Note: This is required in 2010 only.In 2012, app closes when test run finishes.

Use ApplicationUnderTest.CloseOnPlaybackClea

nup & BrowserWindow.CloseOnPlaybackCleanup

to preserve between test methods.

Understand Searching and Filtering http://tinyurl.com/SearchAndFilter

Searching is used to look for all possible controls

Filtering is used to narrow that list to exactly one match

X, Y is only used to guide where the clicks happen within a control

Mouse.Click(uIGOButton,new Point(19, 18));

Employ Databinding

Test → Windows →Test View → Properties →

Select CSV, XML, or Database.

this.UIMap.SimpleTestParams.UIMyTextboxEditText = TestContext.DataRow[“col_name"].ToString();

Note: Databinding to test parameters happensautomatically when converting from manual test cases

Note: This dialog is currently not available in

Visual Studio 2012. Need to hand-edit data connection

attributes.

Make Friendlier Assertions

Makes it easier to analyze test run data

Assert.AreEqual(foo, bar, "Sales tax total is wrong.");

Smile! You’re on Camera

Image pic = this.<top_level_window>.CaptureImage(); //Or for entire desktop: Image pic = UITestControl.Desktop.CaptureImage ();pic.Save(@"c:\file.bmp");TestContext.AddResultFile(@"c:\file.bmp");

Enable Rich Action Logs

QTAgent32.exe.config:<system.diagnostics> <switches>

<add name="EqtTraceLevel" value="4" /> </switches> </system.diagnostics><appsettings> <add key="EnableHtmlLogger" value="true"/> <add key="EnableSnapshotInfo" value="true"/></appsettings>

Microsoft Test Manager → CUIT Workflow Tips

Mark each step as pass/fail when recording manual tests

Carefully consider which test cases make sense for automation, and when

Use the Automation Status field of test cases to identify candidates for automation

Continue on Error

For actions that may not happen every time

Playback.PlaybackSettings.ContinueOnError = true;or

Multiple UI Map Files http://tinyurl.com/MultipleUIMaps

Allows you to separate logical collections of controls

Easier to edit independentlyEasier to version controlRequires a bit more up-front effortDoesn’t play well with MTM workflow

Test API

http://testapi.codeplex.com/Input Injection APIs Command-Line Parsing APIs Visual Verification APIs Combinatorial Variation Generation APIs Managed Code Fault Injection APIs Text String Generation APIs Memory Leak Detection APIs Object Comparison APIsMore…

Coded UI Test Enhanced Frameworkhttp://cuite.codeplex.com/

Simplifies CUIT code

Eliminates need to maintain multiple UI maps

Logical support for tables

Built and supported by the community

Making Your Applications More Testable

Use a testable application stackGive names to controls

Use naming conventions and stick with themIf you need to change the names of controls, refactor them along with your tests

Carefully consider the implications of changing UI flowsEducate non-test developers on CUITsEducate manual testers on CUITs

Playback Settings

Search timeouts

Search behaviors

Error behaviors

See http://tinyurl.com/PlaybackSettings

Windows Automation API 3.0

If running an O/S prior to Windows 7/2008 R2, install this platform update: http://support.microsoft.com/kb/971513/

What About Load Testing?

Coded UI tests assume they have “control” of the mouse and keyboard

Load testing can be accomplished but you need one machine (physical or virtual) per virtual user

See http://msdn.microsoft.com/en-us/library/ff468125.aspx

Environments

SCVMM Environment Standard Environment

Build-Deploy-Test

Snapshot / Restore

Share Bug Snapshots

Network Fencing

3rd Party Virtualization

Physical Machines

demo

Using Lab Managementwith Coded UI Tests

Go get started!

Download the RC: www.microsoft.com/VisualStudio/11 Download the ALM VM: http://aka.ms/VS11ALMVMAsk the Experts, TLC area

tfspreview.com

Related Content

Breakout Sessions

DEV214 Introducing the New Visual Studio 2012 Unit Testing Experience

DEV344 Developer Collaboration with Team Foundation Server 2012

AAP309 Making Agile Estimation Work

DEV337 Software testing with Microsoft Test Manager 2012 and Lab Management

DEV317 Going Beyond F11: Debug Better and Faster with Visual Studio 2012

DEV340 Taking ALM to the Cloud with the Team Foundation Service

AAP313 Scrum Under a Waterfall

DEV362 From Development to Production: Optimizing for Continuous Delivery

DEV363 Dev-Ops Best Practices on the Microsoft Stack

DEV212 Implementing Scrum Using Team Foundation Server 2012

Related Content

Breakout Sessions

DEV346 All Aboard the Team Foundation Server Express

DEV217 Deep Dive into the Team Foundation Server 2012 Agile Planning Tools

DEV365 Advanced IntelliTrace in Production with Visual Studio 2012

DEV316 Application Lifecycle Management Tools for C++ in Visual Studio 2012

DEV343 Implementing Team Foundation Server in the Enterprise

DEV321 Continuous Feedback in Agile Teams

DEV243 Demystifying Team Foundation Server Builds

DEV411 Testing Un-testable Code with Fakes in Visual Studio 2012

DEV410 Deep Dive into Team Foundation Server 2012 Reporting

DEV412 Identify and Fix Performance Problems with Visual Studio 2012 Ultimate

DEV345 The Accidental Team Foundation Server Admin

Related Content

Breakout Sessions

DEV318 Working on an Agile team with Visual Studio 2012 and Team Foundation Server 2012

DEV339 Metrics That Matter: Improving Lean and Agile, Kanban and Scrum

AAP204 Introduction to Kanban

Hands-on Labs (session codes and titles)

DEV11-HOL

Agile Project Management in Team Foundation Server 2012

DEV12-HOL

Build the Right Software and Collaborate Effectively Using Storyboarding and Feedback Tools in Visual Studio 2012

DEV13-HOL

Using IntelliTrace with Production Systems to Quickly Diagnose and Fix Issues

DEV16-HOL

Learn How Microsoft Test Manager 2012 Will Embrace an Exploratory Testing Approach

DEV17-HOL

Explore the New Unit Testing and Code Clone Capabilities of Visual Studio 2012

DEV19-HOL

Discover How the New Features of Team Foundation Server 2012 Can Improve Collaboration in Your Development Team

Resources

Connect. Share. Discuss.

http://europe.msteched.com

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet Resources for Developers

http://microsoft.com/msdn

Evaluations

http://europe.msteched.com/sessions

Submit your evals online

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.