to deploy or not-to-deploy - decide using tta's trend & failure analysis
DESCRIPTION
Here are the slides from my recent talks on "To Deploy or Not-To-Deploy - decide using TTA's Trend and Failure Analysis" More information available from my blog: http://essenceoftesting.blogspot.com/search/label/ttaTRANSCRIPT
TO DEPLOY, OR NOT TO DEPLOY? DECIDE USING TEST TREND ANALYZER (TTA)
Anand Bagmar
Test Practice Lead 1
@BagmarAnand
about.me/anand.bagmar
ABOUT ME
WHAT DO YOU EXPECT FROM THIS SESSION?
WHAT ARE THE CRITERIA FOR DETERMINING BUILD PROMOTION & RELEASE CANDIDATES?
ORGANIZATION OBJECTIVE
Money / Value
Time
Quality
REALITY IN ORGANIZATIONS
Test Automation
A PRACTICE THAT MAKES TEAMS SUCCESSFUL?
Test Automation
A PRACTICE THAT MAKES TEAMS UNSUCCESSFUL?
What is Test Automation?
Automation – A Safety Net
Manual / Exploratory UI
Web Service
View
JavaScript
Integration
Unit
Time
Value
Cost
IDEAL TEST AUTOMATION PYRAMID
Business-facing Tests
Technology-facing Tests
REALITY
UI
Web Service
View
JavaScript
Integration
Unit
ICE-CREAM CONE ANTI PATTERN
Manual / Exploratory
Developers Test Pyramid QA Team Test Pyramid
Web Service
View
JavaScript
Integration
Unit
Manual / Exploratory
UI
Web Service
DUAL TEST PYRAMID ANTI-PATTERN
How does your Test Automation Pyramid look like?
CONTINUOUS INTEGRATION (CI)
”… is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible…."
- Martin Fowler, ThoughtWorks Chief Scientist http://www.thoughtworks.com/continuous-delivery
CI PROCESS
DASHBOARD
SMALL TEAMS
19
HOW ARE YOUR TEAMS STRUCTURED
Product 1 Product 2
Product 3 Product 4
CONTINUOUS DELIVERY (CD)
… is the natural extension of Continuous Integration: an approach in which teams ensure that every change to the system is releasable, and that we can release any version at the push of a button. Continuous Delivery aims to make releases boring, so we can deliver frequently and get fast feedback on what users care about.
- Martin Fowler, ThoughtWorks Chief Scientist http://www.thoughtworks.com/continuous-delivery
CONTINUOUS DELIVERY
IS YOUR PRODUCT IS READY TO GO TO THE NEXT LEVEL?
How do you know?
DOES ALL TESTS PASSING MEAN NO DEFECTS?
Test Trend Analyzer (TTA)
TTA DEMO
Trend Analysis
Failure Analysis
Admin
Upload Data Dashboards
TTA - TREND ANALYSIS
Pyramid View
Comparative Analysis
Test Execution Trends
TTA - FAILURE ANALYSIS
Compare Runs
Failure Analysis
External Dashboards
Other features
38
FOR DIFFERENT SUB – PROJECTS
TEST STATUS FOR A SINGLE PROJECT
FOR TECH-OPS / IT
HOW TO USE TTA?
HOW DOES IT WORK?
COLLECT TEST RUN DATA IN TTA
Manual Upload
Automatic Upload
Job(s) Triggered
Run Tests
Run Tests
Run Tests
CI INTEGRATION
Test Execution Machine
CI Job Triggered
Clean Compile Setup Run Tests Return to CI
Test Execution Command
Test Execution Machine
Test Execution Command
CI Job Triggered
Clean Compile Setup Run Tests Return to CI
Send Test Results to TTA
THE POWER THIS BRINGS TO THE TEAMS
Test Execution Machine
Test Execution Command
Send Test Results to TTA
Ant / Maven / Gradle / Make / Rake …
xUnit / testNG Java / C# / Perl / Python /
Ruby …
GO / Hudson / Jenkins / Bamboo / ..
CI Job Triggered
Clean Compile Setup Run Tests Return to CI
HOW CAN YOU HELP?
FEATURES
USE AND GIVE FEEDBACK
