exploratory testing with jira | qasymphony webinar
TRANSCRIPT
Exploratory Testing + JIRAMay 18 2016
Kyle McMeekin - QASymphony
Join the conversation – use the hashtag #exploratorytesting on Twitter
OUR PRESENTER
Kyle McMeekin, Senior Product Specialist at QASymphony
Kyle McMeekin is a Senior Product Specialist at QASymphony focusing his time on customer demonstrations of product offerings and technical support.
He previously spent time as a tester at Cognizant Technology Solutions and has recently moved to the Atlanta area after growing up near Washington, DC and then spending a few years in the arctic known as Minnesota. Graduate of the University of Michigan, Kyle is excited to watch Michigan win the National Title in 2016.
Join the conversation – use the hashtag #exploratorytesting on Twitter
AGENDA
JIRA’s ET Suggestions
Problems With Traditional Testing
Exploratory Testing (ET) Explained
Tools for ET
Join the conversation – use the hashtag #exploratorytesting on Twitter
JIRA AND EXPLORATORY TESTING
• “As the product grows, the amount of testing grows exponentially–and QA invariably struggles to keep up”
Join the conversation – use the hashtag #exploratorytesting on Twitter
https://www.atlassian.com/landing/software-testing?tab=exploratory-software-testing
JIRA AND EXPLORATORY TESTING
• You want exploratory testing to be product focused
• You need a thorough way of capturing test evidence… print screen to Paint to an email/doc no longer works
• Exploratory testing is not just for Testers
Join the conversation – use the hashtag #exploratorytesting on Twitter
https://www.atlassian.com/landing/software-testing?tab=exploratory-software-testing
WHY SHOULD WE DO ET?
Controlled study link:: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3696&rep=rep1&type=pdf
Testing with test cases vs. exploratory
testing take almost 7x longer
7xTIME SAVED MORE BUGSTesting with test cases
vs. exploratory testing finds more
defects
ACCURACYTesting with test cases
vs. exploratory testing finds more
defects
Join the conversation – use the hashtag #exploratorytesting on Twitter
PROBLEMS IN TRADITIONAL TESTING
Prescribing detailed test steps, and test planning, can cloud
interactions.
Focus on compressive documentation blocks cognitive thinking abilities
Customer collaboration in testing remains in silo or
nonexistent. Comprehensive test plans create a
tunnel vision affect
Individuals and interactions over processes
and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Join the conversation – use the hashtag #exploratorytesting on Twitter
PROBLEMS IN TRADITIONAL TESTING
Function A
Function B
Function C
Function D
• Step1• Step2
• Step1• Step2
• Step1• Step2
• Step1• Step2Scripted Testing
Function A
Function B
Function C
Function D
Exploratory Testing
• Objective1
• Objective2
Join the conversation – use the hashtag #exploratorytesting on Twitter
WHAT IS EXPLORATORY TESTING?
1. Parallel test planning, test design, and test execution
2. Specific yet flexible
3. Aligned towards investigation of potential opportunities
4. Values depth and attention to detail during testing
5. Fosters knowledge sharing and accountability
Join the conversation – use the hashtag #exploratorytesting on Twitter
PARALLEL PLANNING, DESIGNING, EXECUTION
Unlike traditional testing techniques, planning, design, and execution happen concurrently, allowing efficiencies of time as well as flexibility in approach
PLAN
DESIGN
EXECUTE
REPORT
PLAN
DESIGN
EXECUTE
REPORT
Traditional Scripted Testing Exploratory Testing
Join the conversation – use the hashtag #exploratorytesting on Twitter
Exploratory Testing provides a specific lends through which to perform testing – whether that be a user person, functionality, criteria (i.e. Localization), ect.
However, it allows testers to use the tool as an end user would, not necessarily as the product owner envisioned it.
SPECIFIC YET FLEXIBLE
I tested the application as the end user would
I tested the application as the script prescribed
Join the conversation – use the hashtag #exploratorytesting on Twitter
INVESTIGATE OPPORTUNITIES
Exploratory testing rewards testers who identify unknown areas of “opportunity” within the application, as they are essential in maintaining a backlog of future test charters
Traditional Scripted Testing
Exploratory Testing
Join the conversation – use the hashtag #exploratorytesting on Twitter
EXPLORATORY KNOWLEDGE SHARING
Exploratory testing relies on knowledge sharing to reach full potential – developing testers who understand the impact of more areas of the application allows them to identify more areas of risk and opportunity
Plan
DesignExecute
Report Transfer Learning
Example Questions to Ask
• Have you seen this before?
• What am I not considering?
• Why would someone do this?
• How would you have tested this?
Join the conversation – use the hashtag #exploratorytesting on Twitter
BRINGING IT ALL TOGETHER: CAR SHOPPING
Scripted Test Drive Exploratory Test Drive
Join the conversation – use the hashtag #exploratorytesting on Twitter
WHAT EXPLORATORY TESTING IS NOT
1.Exploratory testing is not unstructured
2.Exploratory testing is not the only type of testing
Join the conversation – use the hashtag #exploratorytesting on Twitter
Successful exploratory testing still contains parameters such as:
- A goal of the exploration
- A log of the activity performed
- A lens through which the testing is performed (user persona)
Performing exploratory testing without involving some parameters such as the above allows a greater risk of unsuccessful implementation of exploratory testing
EXPLORATORY TESTING UNSTRUCTURED TESTING
Join the conversation – use the hashtag #exploratorytesting on Twitter
HOW TO STRUCTURE ET: SESSION BASED
Session Based Test Management is a popular framework, because it tracks important data on the testing, while also adding some much needed structure.
• Session charter (includes a mission statement, and areas to be tested)
• Tester name(s) • Date and time started • Task breakdown (the TBS metrics) • Data files • Test notes • Issues • Bugs
Session Report
More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf
Join the conversation – use the hashtag #exploratorytesting on Twitter
EXAMPLE: SESSION REPORTDon’t forget the debrief!
A debrief is a short discussion between the manager and tester (or testers) about the session report. Jonathan Bach uses the acronym PROOF to help structure his debriefing. PROOF stands for:
• Past: What happened during the session?
• Results: What was achieved during the session?
• Obstacles: What got in the way of good testing?
• Outlook: What still needs to be done?
• Feelings: How does the tester feel about all this?
More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf
Join the conversation – use the hashtag #exploratorytesting on Twitter
UAT WITH EXPLORATORY TESTING
UAT Challenge ET Benefit
UATer’s are unfamiliar test case syntax and need continual clarification
Allow UATer’s to perform the business flows they know well without test scripts
UATer’s are not trained on test case management, automation tools, etc.
Focus UATer’s time on learning how to document proper defects, reduce time to ramp
UATer’s have a shorter attention span – they are not used to testing 6-8 hrs. per day
Allow UATer’s to veer off the rails from time to time and investigate areas of interest
UATer’s have a short period of time in which to provide feedback
Ensure that as much of the UATer’s time as possible is dedicated to ET
Join the conversation – use the hashtag #exploratorytesting on Twitter
Keep your Best Testing Talent
IMPROVE JOB SATISFACTION MOVE FROM TESTER TO SME
WHY DO EXPLORATORY TESTING?
Join the conversation – use the hashtag #exploratorytesting on Twitter
TOOLS FOR THE JOB
Join the conversation – use the hashtag #exploratorytesting on Twitter
JIRA CAPTURE
JIRA Capture• Capture screen shots• Annotate and share• Create and record
exploratory testing sessions
• Submit issues in to JIRA
Join the conversation – use the hashtag #exploratorytesting on Twitter
qTest Explorer• Applicable on any
application; including mobile, desktop, and web
• Automatically generate TCs from your sessions
• Capture environment details
• All sessions stored in central repository
• Submit test cases and defects directly to qTest or JIRA.
QTEST EXPLORER
Join the conversation – use the hashtag #exploratorytesting on Twitter
REFERENCES / THOUGHT LEADERS
James Bach - http://www.satisfice.com/
Jonathan Bach - https://jonbox.wordpress.com/
Michael Bolton - http://www.developsense.com/
Paul Holland - http://testingthoughts.com/blog/author/testthought
Keith Klain - http://qualityremarks.com/
Brian Osman - https://bjosman.wordpress.com/
Exploratory Testing White Board Friday Series by QASymphony
Join the conversation – use the hashtag #exploratorytesting on Twitter
Thank you