case procountor: zephyr test tool deployment
TRANSCRIPT
![Page 1: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/1.jpg)
![Page 2: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/2.jpg)
Zephyr Test Tool Deployment | Procountor Knowit Oy, Jussi Rautio 6.10.2015
PAGE 2 [CONFIDENTIAL] THIS IS KNOWIT 6.10.2015 www.knowit.fi
![Page 3: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/3.jpg)
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 3
Senior Quality Consultant, Service Productization Responsible, Service Manager, Test Tool Technical support, Test Tool Deployments and Migrations…
• Speaks, train, coach and mentor about
• How to take new testing tool in use and best practices.
• User trainings for testers, test managers, product owners and quality managers
• Metrics and KPI`s benchmarking and design
• Data Quality
• Process enhancements
• Etc.
JUSSI RAUTIO, KNOWIT
www.knowit.fi
![Page 4: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/4.jpg)
KNOWIT
Scandinavian company that is listed on Nordic
Exchange Stockholm
Knowit offering in Finland:
• Test and quality management
• Solutions for digital business and information
management
• Development and maintenance of client-
specific applications
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 4 www.knowit.fi
![Page 5: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/5.jpg)
02.09.15 [CONFIDENTIAL] THIS IS KNOWIT PAGE 5
![Page 6: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/6.jpg)
• Leading provider of cloud accounting software
• Used by over 10 000 companies and over 400
accounting offices
• Available in Finland, Norway, Sweden and
Denmark
02.09.15 [CONFIDENTIAL] THIS IS KNOWIT PAGE 6
![Page 7: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/7.jpg)
PROCOUNTOR ATLASSIAN ENVIRONMENT
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 7
www.ambientia.net 7
Crowd
JIRA Fisheye Confluence
Refi
ne
d w
iki
JIR
A a
gile
Zep
hyr
for
JIR
A
Tim
es
hee
t re
po
rts
Cru
cib
le
Confluence • Social Intranet
• Knowledge Management
JIRA • Requirement management
• Product development (scrum)
• Test management
Fisheye/Crucible • Source code browsing
• Code review
Crowd • User management
*500
*50 *100 *250
Hosted by: www.ambientia.net
www.knowit.fi
![Page 8: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/8.jpg)
10/6/2015 © Copyright Knowit Oy 2014 | Confidential | Version 1.0 8
Problem Description
www.knowit.fi
![Page 9: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/9.jpg)
PROCOUNTOR NEED
• Procountor is a rapidly growing firm
• New people are hired constantly
• Testing tool and processes need to be updated to meet the new demands
• Knowit was asked to enhance Procountor testing
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 9 www.knowit.fi
![Page 10: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/10.jpg)
TEST PROCESS ANALYSIS
Knowit test process analysis for Procountor
Means of analysis were:
• Workshops
• Interviews
Two weeks after the release development teams are very busy on fixing bugs. Why?
• Too tight schedule
• Fixed content
• Not enough time for regression testing
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 10 www.knowit.fi
![Page 11: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/11.jpg)
TEST PROCESS ANALYSIS CONCLUSION
Insufficient regression testing time ->
Whole team approach should be introduced ->
Testing tasks are in the different testing tool ->
Better tool to enforce task sharing ->
Jira ensures already a common way of working ->
Test tool Zephyr was chosen to be used in regression testing
because of seamless integration to Jira among other things
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 11 www.knowit.fi
![Page 12: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/12.jpg)
© Copyright Knowit Oy 2014 | Confidential | Version 1.0 12
Introducing the Test Tool
www.knowit.fi
![Page 13: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/13.jpg)
TEST TOOL DEPLOYMENT PRINCIPLES
Fail fast
• If tool is not suited for you, don’t waste your time, look other way
Start light
• Put only necessary test data in to system in pilot phase
• Do only minimum tweaking to the tool at the beginning
Embrace the change
• Give tool a chance
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 13 www.knowit.fi
![Page 14: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/14.jpg)
www.knowit.fi
Enhanced
Quality
and
Effectivity
Create a usage model for test tool
Test tool in pilot use for regression testing
Usage model tweaking
Test tool in use for wider testing
Test tool in use internationally
Continuous improvement activities
Integrations to other systems
Test tool used through organization if Finland
Virtualization & automation
![Page 15: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/15.jpg)
GENERAL ABOUT ZEPHYR
• Brings test management and execution functionalities to Jira
New issue type Test
• Possibility to have as an add on for Jira or as an own product
• First Version released in the beginning of 2012
• Latest version 2.6.0 Released on 09.2015
Characteristics
• Easy to use for Jira users
• Customizable
• No test case version control
• If lots of add-on’s are in use the update process of Jira and Zephyr becomes challenging
• No folder structure
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 15 www.knowit.fi
![Page 16: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/16.jpg)
THINGS TO DO BEFORE DEPLOYMENT 1/3
• Fields for tests must be agreed
• default fields can be used at start
• Screens must be agreed
• Tests workflow should be done when testing process with the new tool is mature enough
• User groups and rights should be put in order
• For example who can modify test executions should be restricted
• Procountor has one tester per group so there is no need to restrict user rights
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 16 www.knowit.fi
![Page 17: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/17.jpg)
THINGS TO DO BEFORE DEPLOYMENT 2/3
Labels must be decided
• Label is users self chosen tag for tests
• Labels should be agreed so that their amount would not explode
• Labels could be for example: Functional testing, Regression testing, Usability, Load…
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 17 www.knowit.fi
![Page 18: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/18.jpg)
THINGS TO DO BEFORE DEPLOYMENT 3/3
Components must be decided
• Components are user defined parts of software
• For example: Accounting, User Management, Billing..
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 18 www.knowit.fi
![Page 19: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/19.jpg)
TEST STATUSES
One part of test process definition is specifying test statuses
• Statuses can be modified based on test process
Default test statuses of Zephyr are
• PASS
• FAIL
• WIP
• BLOCKED
• UNEXECUTED
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 19 www.knowit.fi
![Page 20: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/20.jpg)
LINKING TESTS TO USER STORIES
Tests can be linked to User Stories
• “Is blocked by” -link statuses are NOT affecting if the test can be run or not
Linking should be done always so changes in User Story can be followed easily
• Changes in user story are NOT sending signal for re-testing
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 20 www.knowit.fi
![Page 21: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/21.jpg)
TEST CASE DIVISION
1. By Project
• Project is a way to division test cases in a very high level. Procountor has only one Project
2. By Version
• Fix Versions –field can be used to division tests. Test Summary Screen shows tests by version
automatically
3. By Component
• Test can be attached to a component (multivalue field.) Test Summary shows also these.
4. By Label
• Anyone can create new labels so this should be agreed in organization to keep it controlled
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 21 www.knowit.fi
![Page 22: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/22.jpg)
CYCLES – TEST ROUNDS
Tests are executed in Cycles and one test can be in many Cycle
Tests can be taken to test Cycles either by:
• One at a time
• By Filter
• By Component
• By Label
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 22 www.knowit.fi
![Page 23: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/23.jpg)
NAMING AND DIVISION OF CYCLES
Cycle name should present their division and support reporting.
Cycle names can be for example:
• Regression test for version xx (Procountor)
• Sprint
• Week
• Tester
• Group
• Combination of above
…
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 23 www.knowit.fi
![Page 24: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/24.jpg)
© Copyright Knowit Oy 2014 | Confidential | Version 1.0 24
Monitoring Test Tool Deployment Success
www.knowit.fi
![Page 25: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/25.jpg)
TEST TOOL CONCEPT OWNER
Someone from the organization should be named as a
test tool concept owner after the test tool deployment
Test tool concept owner tasks:
• Follow test tool efficiency continuously
• Follow test tool development roadmap
• Introduce new test tool features to organization
• Gather user requests for test tool customization and integrations
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 25 www.knowit.fi
![Page 26: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/26.jpg)
MEASURING OF TEST TOOL EFFICIENCY
Test tool efficiency should be followed in during the long period of time
Has the quality of the product enhanced?
• Number of bugs found in production
• Customer satisfaction
Has the efficiency increased?
• Is the tool really used?
• What is the gut feeling? Has the tool helped testing? Is regression testing easier now?
• How wide has the tool usage spread?
• Has the hurry after the release decreased?
• What is the usability of the tool?
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 26 www.knowit.fi
![Page 27: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/27.jpg)
FUTURE SCENARIO FOR PROCOUNTOR
Testing during development could be done with exploratory testing
• Link exploratory testing session to User Story
• Leaves a trail what has been already tested
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 27
• Actor: intended user of the system
• Purpose: the theme of the charter including what particular objective the actor wants to
achieve, i.e., the test conditions
• Setup: what needs to be in place in order to start the test execution
• Priority: relative importance of this charter, based on the priority of the associated user story or
the risk level
• Reference: specifications (e.g., user story), risks, or other information sources
• Data: whatever data is needed to carry out the charter
• Activities: a list of ideas of what the actor may want to do with the system and what would be
interesting to test (both positive and negative tests)
• Oracle notes: how to evaluate the product to determine correct results
• Variations: alternative actions and evaluations to complement the ideas described under
activities
www.knowit.fi
![Page 28: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/28.jpg)
EXPLORATORY TESTING – TEST SESSION
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 28
Session based testing aims to combine accountability and exploratory testing
Test sessions:
• Based on charter
• Uninterrupted session from 60 to 120 minutes
• Evaluation of the functionality or characteristics
• Each session is reported in detail
Test Design
Test Execution
Test Results
Learn Test Charter
Test Session
Test Design
Test Execution
Test Results
Learn Test Charter
Test Session
Test Design
Test Execution
Test Results
Learn Test Charter
Test Session
Session-Based Test Management
www.knowit.fi
![Page 29: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/29.jpg)
TOOLS FOR EXPLORATORY TESTING
Record Steps to Reproduce a Problem
• Windows Out Of the Box tool to take screenshots
• Creates screenshots on each click and typing and creates one file from the recording
• Free in all Windows versions
JIRA Capture
• Web –application to create issues easily
• Records Test Session length, issues and tester notes
• Needs to be purchased
6.10.2015 © Copyright Knowit Oy 2015 | Confidential | Version 1.0 29 www.knowit.fi
![Page 30: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/30.jpg)
THANK YOU!
QUESTIONS?
www.knowit.fi
![Page 31: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/31.jpg)
Knowit Oy Tehtaankatu 27-29 D 00150 Helsinki tel. +358 40 3400600 www.knowit.fi
![Page 32: Case Procountor: Zephyr test tool deployment](https://reader033.vdocuments.site/reader033/viewer/2022052405/58ab207e1a28abb5278b54bd/html5/thumbnails/32.jpg)
What Test Management challenges and
possibilities are there in the agile/DevOps
model?
www.ambientia.net
Let’s talk!
© Copyright Knowit Oy 2015 | Confidential | Version 1.0 www.knowit.fi