james a. whittaker a. whittaker software architect visual studio team test microsoft...

James A WhittakerJames A WhittakerSoftware ArchitectSoftware ArchitectVisual Studio Team TestVisual Studio Team TestMicrosoft Microsoft

httpblogsmsdncomjames_whittakerjwmicrosoftcom

It wasnrsquot until 2003 that the first exoplanet was confirmed to orbit the star Gamma Cephei

By 2006 Astronomers confirmed the existence of over 200 exoplanets

As of February 2009 that number has climbed to 340

Global climate change

Alternative energy

Managing global finance

bull ldquoA second V-22 crash in December 2000 killed four Marines The accident was pegged to a ruptured hydraulics line and a software failurerdquo - httpcommunityseattletimesnwsourcecomarchivedate=20030226ampslug=osprey2 6

bull ldquoFans of the rock band Pearl Jam got a taste of cyber-gridlock Saturday when a software failure disabled phone lines set up for ticket requests for shows in Seattle Toledo Ohio and other citiesrdquo - httpcommunityseattletimesnwsourcecomarchivedate=19960814ampslug=234416 1

bull ldquoPatriot Missiles Tragic Failure Might Have Been Averted -- Computer Glitch Probably Allowed Scud To Avoid Intercept Army Saysrdquo - httpcommunityseattletimesnwsourcecomarchivedate=19910815ampslug=130007 1

bull ldquoSoftware Failure Halts Big Board Trading for Over an Hourrdquo - httpquerynytimescomgstfullpagehtmlres=9C05E0D7173EF93AA35755C0A967 9C8B63ampscp=1ampsq=22software20failure22ampst=cse

bull ldquoThis is what happened in 2003 when a software failure near Toledo Ohio combined with sagging power lines and a simple error by a control room worker to cause a blackout in eight Northeastern and Midwestern states and Ontariordquo - httpwwwnytimescom20070708opinionnyregionopinions08CIperrowhtml_r=1 ampscp=14ampsq=22software20failure22ampst=cseamporef=slogin

bull ldquoNike last year had to reduce prices and fly some products to customers after a software failure resulted in excess inventory and missed shipmentsrdquo - httpquerynytimescomgstfullpagehtmlres=9407E2DB133EF93BA15755C0A964 9C8B63ampscp=24ampsq=22software20failure22ampst=cse

Pappa er

vi der

snart

bull Developers must do better unit testingndash The little things must be right and fixed immediatelyndash What bugs are best suited for unit testing

bull SDETs must focus automation correctlyndash Large regression repository v special purpose ldquosmart

bombsrdquondash What bugs are best suited for automation

bull Resuscitate the manual testerndash Key scenarios subtle bugs require a brain-in-the-loopndash But where does the guidance come from

bull Manual testers face a lot of decisionsndash Which input environment data storendash Which feature path scenario

bull What information helps us make the right choices

bull What guidance helps us prioritize tasks and improve our techniques

bull Gaming is hard game worlds are complexbull Navigating the game requires decision

makingndash Which areas to investigatendash Where is the useful stuffndash How do I stay alive progress through the levels

bull Being a skilled gamer requires ndash Free flowing information from the gamendash Technique know what works and what doesnrsquot

bull We have to know what to do with itndash Manual testers need guidancendash Teams need institutional memory

bull Stop defining bugs by their repro steps and start generalizing them to the revealing technique

bull Tourism is confusing destinations are complex

bull Deciding what to visit and how to get therendash What are the major attractionsndash How do I see as much as possible during my

short vacationndash Where are other tourists going

bull Being a skilled tourist requires ndash Good maps and guidebooksndash Easy to follow advice

bull VSTS exploratory testing SWAT teamndash Jim Shobe David Gorena Elizondo Hiromi

Nakura Vessie Djambazova Ratnaditya Jonnalagadda Shay Mandel

bull Microsoft exploratory testing working groupndash Geoff Staneff David Gorena Elizondo Nicole

Haugen Shawn Brown Bola Agbonile Jeff MacDermot Jeremy Croy Rob Straavaldson

bull A sample of the toursndash Money tour test the features that users buy the app forndash Rained-out tour start and stop tasks hit cancelndash Obsessive compulsive tour do tasks multiple timesndash Back alley tour test the least used featuresndash All nighter tour never close the app

bull Specific focused testing advice ndash Guide the tester through paths of the softwarendash Help a tester think through inputs data and

environment settings that satisfy the goals of the tourndash A more methodical and purposeful way to test

bull Tours as exploratory test guidance and a way of partitioning test concernsndash Each tour is guidance about testing a very specific type

of application capabilityndash Tours collectively encompass all interesting scenariosndash Tours can be associated with the component-feature-

capability map to tie testing effort with specific test needs

bull The ldquotouring testsrdquo take the discussion of how to test up to a higher levelndash Instead of talking about specific test cases they guide

us toward talking about test conceptsndash Such higher level concepts have been missing from our

testing vocabulary for far too long

bull Want to test the interface between a database component and the application Run the FedEx Tour

bull Want to test legacy code Run the Museum Tour and the Prior Version Tour

bull Want to test how two features interact Run the Guidebook Tour between them

bull Want to find UI bugs Run the Supermodel Tourbull Want to find stress bugs Run the Saboteur (get it)

and the TOGOF Tourbull Want to find timing bugs Run the Rained Out Tour

MAIN SHELL

Context (TFS)

- Add a new TFS Server

- Connect to a TFS Server

- Delete a TFS Server

Open Items

- Verify that only savable activities show up here

- Verify that you can navigate to them

TESTING CENTER

Plan

- Contents

Add new static suites

Add new query-based suites

hellip

Refresh

- Properties (current test plan)

Modify iteration

Modifycreate new manual test setting

hellip

Modify startend dates

- Plan Manager

Select a current test plan

Create a new test plan

hellip

Refresh test plan manager

TESTING CENTER cont

Test- Run Tests

Run all tests on a suite (automated andor manual or mixed)Run tests with a specific environment chosenhellipMicrosoft Test Runner

- Analyze test runsOpen a test run (automatedmanualmixed)

- Change title of a test run- View summaryresults overviewtestsattached files- hellip- Context menus for actions mentioned

hellipContext menus for actions mentioned

- My bugsView bugs assigned to meCreate a new bughellipCreate a copy of the bug

James A Whittakerhttpblogsmsdncomjames_whittaker

jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30

It wasnrsquot until 2003 that the first exoplanet was confirmed to orbit the star Gamma Cephei

By 2006 Astronomers confirmed the existence of over 200 exoplanets

As of February 2009 that number has climbed to 340


Alternative energy








Pappa er

vi der

snart



































MAIN SHELL

Context (TFS)




Open Items



TESTING CENTER

Plan

- Contents



hellip

Refresh


Modify iteration


hellip


- Plan Manager



hellip


TESTING CENTER cont

Test- Run Tests







jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30


Alternative energy








Pappa er

vi der

snart



































MAIN SHELL

Context (TFS)




Open Items



TESTING CENTER

Plan

- Contents



hellip

Refresh


Modify iteration


hellip


- Plan Manager



hellip


TESTING CENTER cont

Test- Run Tests







jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30








Pappa er

vi der

snart



































MAIN SHELL

Context (TFS)




Open Items



TESTING CENTER

Plan

- Contents



hellip

Refresh


Modify iteration


hellip


- Plan Manager



hellip


TESTING CENTER cont

Test- Run Tests







jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30



































MAIN SHELL

Context (TFS)




Open Items



TESTING CENTER

Plan

- Contents



hellip

Refresh


Modify iteration


hellip


- Plan Manager



hellip


TESTING CENTER cont

Test- Run Tests







jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30

TESTING CENTER cont

Test- Run Tests







jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30


jwmicrosoftcom


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30


jwmicrosoftcom

Slide Number 1

Slide Number 2

Slide Number 3

Slide Number 4

Slide Number 5

Slide Number 6

Slide Number 7

Slide Number 8

Slide Number 9

Slide Number 10

Slide Number 11

Slide Number 12

Slide Number 13

Slide Number 14

Slide Number 15

Slide Number 16

Slide Number 17

Slide Number 18

Slide Number 19

Slide Number 20

Slide Number 21

Slide Number 22

Slide Number 23

Slide Number 24

Slide Number 25

Slide Number 26

Slide Number 27

Slide Number 28

Slide Number 29

Slide Number 30

james a. whittaker a. whittaker software architect visual studio team test microsoft...

Documents