State of testing @ MicrosoftQuality and collaboration throughoutthe development lifecycle

Robert MacLeanSenior DeveloperALM Ranger@rmaclean

Hi

Happy BTTF Day!

Sorry, this talk contains no hover boards

ALM RangersPassionate about ALMStaff & Passionate experts creating open source solutions and guidance for Microsoft’s ALM products. Solutionshttp://aka.ms/vsarsolutionsTest Planning & Management GuideVisual Studio Test Tooling Guides: MTM, CodedUI, Fakes & IntelliTraceDevice Testing Guidance, HOL & PostersWCF Load Test

Changing WorldOur Development VisionTools to empower Test, Feedback & MonitoringGet the right people, the right tools

Topics

Changing world01

Practices: Waterfall to Agile to DevOpsSystems: Heterogeneous ecosystems and BYOD Cloud & Hybrid-Cloud: from “never” to “maybe” to ”when”

Changing World

Plan1 Monitor +

Learn

Release

Develop + Test2

Development

Production

DevOps4

3

Source controlAgile planning

Test case managementBuild automationContinuous deploymentRelease

managementLoad testingFeedback

managementTeam collaborationApplication

telemetryLab management

Team Foundation Server

Visual Studio Online

Our development vision02

Microsoft development platform

Runtimes + Languages.NETC# + VBC++HTML/JS

Tools

Cloud

Devices

AndroidiOSWindows

Our Focus

Every Team MemberEvery App

Every app …

Every team member …

Enterpriseteams

Individual developers

Tools to empower Test, Feedback & Monitoring03

Put quality at the center of everything you do

Agile test planning and executionMonitor & share quality status

Test CaseManagementStreamline qualityTake advantage of a toolset optimized for the needs of testers, giving them flexibility in how they work while at the same time keeping them in sync with the rest of the teamImproved web-based Test Case Management helps testers author, manage and execute test cases using any modern web browserMicrosoft Test Manager further empowers your testers by giving them comprehensive tools for automated and manual testing in a desktop application optimized for the needs of today’s agile QA professional

Manual & Exploratory TestingRun, record, play backCapture detailed records of steps performed, behaviors observed, and the status of each test step with the fully-instrumented and configurable Test Runner. Pause testing at any time to report a bug, even if it’s not directly related to the current test caseHigh-fidelity defect reports go directly to the development team, including all the technical data needed to reproduce and resolve the error quickly. Record tests for later playback to speed up test sessions and enable test automation

Browser-basedTestingTesting in the browserRun manual tests and record test results for each test step using a toolset optimized for the needs of testersThe web-based test runner enables pass-fail results, tracking of test steps, rich commenting, and bug reporting capabilities

FeedbackManagementActionable feedbackOnce you have working software, you're ready to get feedback from your stakeholders. You can ask reviewers to provide videos, screenshots, type-written comments, and ratingsTheir feedback is captured into work items that you can review and use to create a bug or suggest a new backlog item

QualityDashboardsMonitor quality statusBuild dashboards that provide visibility to your team and stakeholders. Stop worrying where to find things and instead use Visual Studio Online or TFS to get everyone on the same page

Manage complexity and close the loop between

Dev and OpsLab managementContinuous deliveryEnterprise DevOps

Lab Management

Dev/Test lab environmentsAutomate your virtual lab environment with Visual Studio Lab ManagementUse templates to define/launch environment configurations for consistent testingProvision dev/test lab environments quickly for testers and developers

ReleaseManagementShip better apps, fasterVisual Studio Release Management makes it easy to automate deployment and testing of your applications in multiple environmentsTeams can create release definitions and automate deployment in repeatable, reliable ways while tracking simultaneous in-flight releasesIncorporate automated functional tests or manual approvals in a release definition to automate the release process all the way to production

Automate The Release Cycle

DEV QAINT PRODTFS

Automates Deployments

Automate The Release Cycle

DEV QAINT PRODTFS

Automate the approval workflow

Automate The Release Cycle

DEV QAINT PRODTFS

Deploy the same way to all stages

Automate The Release Cycle

DEV QAINT PRODTFS

Full Traceability

Release Management

Release Management

Public Cloud

Private Cloud

PowerShell DSC Microsoft System CenterRun Books

SCVMM Azure Pack Azure VMWare

AWS

ApplicationInsightsUsage and performanceGet a 360° view of your app that includes availability, performance and user behaviorUse a powerful and easy-to-use experience to track and improve the success of your applicationAnalyze data across dimensions and make informed decisions about your product’s future

Azure Dev/Test LabUse the cloud for your labEasily construct environments with different artifactsPay for what you useEasy integration into the development tools

Preview

Azure Performance TestUse the cloud to test the cloudEasy access to the tools, so the right people can run the testsPay for what you useTest performance from multiple locations around the world

Preview

Get the right people, the right tools

User Acceptance TestingIncluded in the VSO Basic LicenseTest Pro LicenseIncluded in the VSO Advanced License

StoryboardingFree for everyone

Cloud or OnPremEach VSO license includes the same OnPrem license

http://www.VisualStudio.comhttp://bit.ly/DevTestInDevOpshttp://bit.ly/DevTestLabhttp://blogs.msdn.com/visualstudioalmrangershttp://aka.ms/AppFactoryZAhttp://www.SADev.co.za

Resources