udvid din test portefølje med coded ui test og cloud load test

#CampusDays

Udvid din test portefølje med Coded UI test og Cloud Load testVS20011

Peter Lindberg - Consultant @ Alterate

pli@alterate.dk

@peterlindbergdk

www.linkedin.com/in/peterlindbergdk

www.alterate.dk

#C

am

pusD

ays

Topics

Test automation

Microsoft Test Manager and manual tests

Coded UI Test

Load Test in the cloud with Visual Studio Online

#C

am

pusD

ays

Why test?

We all make mistakes!

But also:

• Are we meeting the requirements and expectations?

• Does the hardware support the application?

• Does it work as expected?

The sooner the defects are discovered – the cheaper they are to fix

#C

am

pusD

ays

Why automate tests?

Cost reduction!

• Manual test is expensive

• Let manual testers spend the time on value adding tests• New features

• Exploratory testing

• “Time to market” – higher release cadence

• Safety as developer

#C

am

pusD

ays

Automated

UI Test

Integration

Test

Unit Test

Exploratory

testing

#C

am

pusD

ays

#C

am

pusD

ays

Microsoft Test Manager and manual tests

#C

am

pusD

ays

Visual Studio Application Lifecycle Management

#C

am

pusD

ays

Microsoft Test Manager (MTM)

Separate application

Fully integrated with Team Foundation Server• Test artifacts are Work Items in TFS

• Links to Product Backlog Items, Bugs etc.

• Queries

• Reporting

• Ties directly in to the developer world

#C

am

pusD

ays

Microsoft Test Manager (MTM)

A fully-instrumented, configurable test runner captures detailed records of steps performed, behaviors observed, and the status of each test step. Pause testing to report a bug, even if it’s not directly related to the current test. High-fidelity defect reports go directly to the development team, including all the technical data needed to reproduce and resolve the error. Record tests for later playback to speed test sessions and enable automation. (visualstudio.com)

#C

am

pusD

ays

What’s in it for the developers?

“captures detailed records of steps performed, behaviors observed, and the status of each test step”

“Pause testing to report a bug, even if it’s not directly related to the current test”

“High-fidelity defect reports go directly to the development team, including all the technical data needed to reproduce and resolve the error”

“Record tests for later playback to speed test sessions and enable automation”

#C

am

pusD

ays

MTM Work Item Structure

#C

am

pusD

ays

Demo

#C

am

pusD

ays

Coded UI tests

#C

am

pusD

ays

What are Coded UI Tests?

Automated tests interacting with the application through the user interface

For example used for testing

• Input validations

• Calculations

• “Shopping cart” functionality

• UI Elements

• Navigation

• ….

#C

am

pusD

ays

Supported configurations and platforms

Configurations

• Windows 7, Windows 8, Windows Server 2008 R2

• x86 and x64

• .NET 2.0, 3.0, 3.5, 4.0 and 4.5

Platforms

• Windows Phone Apps - (Only WinRT-XAML based Phone apps are supported)

• Windows Store Apps - (Only XAML-based Store apps are supported)

• Windows Forms + WPF

• IE 8, 9 and 10 - (9 and 10 only supported on the desktop)

• Chrome (recording not supported, only playback)

• Silverlight 5 (only supported with unsupported plugin)

• SharePoint, Dynamics web client – Fully supported

#C

am

pusD

ays

#C

am

pusD

ays

Test Automation

Test

Execution

Test

Script

#C

am

pusD

ays

Which tests to automate?

Is it often used functionality?

Does the UI change often?

Is the test code easy or difficult to maintain?

How often is the test expected to be run?

#C

am

pusD

ays

Coded UI test features

• Fully integrated into Visual Studio ALM

• Automatic code generation

• “Intelligent” Intent Aware Recording and Resilient Playback

• Execute on build server

• C# / VB.NET

#C

am

pusD

ays

Generate Coded UI Tests

• Automate steps from MTM action recordings

• Coded UI Test Builder

• Write them yourself in C# / VB.NET (advanced)

#C

am

pusD

ays

The generated code

TestMethod

#C

am

pusD

ays

UIMap

#C

am

pusD

ays

Coded UI Test Editor

#C

am

pusD

ays

Coded UI Test Editor

• Change names of test methods and UI elements

• Split steps in to new methods

• Move generated code to UIMap partial class for customizing

• Add pauses

• View and change properties

• Remove unwanted steps (duplicates etc.)

#C

am

pusD

ays

Demo

#C

am

pusD

ays

Load Test in the cloud with Visual Studio Online

#C

am

pusD

ays

#C

am

pusD

ays

What do we want to achieve with load tests?

• Will the planned platform support the expected load and peak loads?

• Will simultaneous users cause problems

• Server load (max. CPU / memory consumption etc.)

• When does it break and how?

Planning is key! Dialogue with Product Owner / stakeholder etc.

What are targets and objectives?

What are the success & acceptance criteria?

Concrete and measurable!!

There’s a big difference between 1 and 5 seconds when a web site load…

#C

am

pusD

ays

How?Generate

traffic to

application

Does it

perform as

expected?

Recreate the

performance

problems

Fix the

performance

problems

#C

am

pusD

ays

#C

am

pusD

ays

So when should I start planning and execute load teste?

Start early!

=

Save time

#C

am

pusD

ays

What can be included in load tests?

• Web test

• Unit tests

• Coded UI (possible – but not recommended)

Customize!

• Coded Web Performance test

#C

am

pusD

ays

#C

am

pusD

ays

To the Cloud!

• Load tests can be executed from the cloud with Visual Studio Online

• MSDN subscribers have 15,000 virtual user minutes included pr. month

• No need to set up a big test rig / infrastructure “on premises”

• Setup in Visual Studio – execute in Azure

• Visual Studio connected to Visual Studio Online

#C

am

pusD

ays

Visual Studio

Load test

web service

Result

database

Controller

Test agent pool

Agent Agent Agent

Agent Agent Agent

#C

am

pusD

ays

Demo

#C

am

pusD

ays

Q&A#Ask me about everything!

Join me at the Microsoft Booth the next 30 minutes @Meet The Experts

Dont forget to: Evaluate this session!

EVENT SPONSORER

EXPO SPONSORER

TRACK SPONSORER