continuous testing in agile projects 2015
TRANSCRIPT
![Page 1: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/1.jpg)
!"
#
$
%
&'
$./run
{
Created by Fabrício Epaminondas ©2015-2020
ContinuousTESTING
Testing in an agile environment
![Page 2: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/2.jpg)
• Continuous Testing • Agile principles • Why Continuous Testing? • Creating your first test project in minutes • How it works • Test process • Effective Continuous Testing
15/10/2015 - 20:30 Location: Cinema didático - Facisa
Agenda
![Page 3: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/3.jpg)
The goal
AGILE PRINCIPLES
![Page 4: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/4.jpg)
The Big Picture of General SDLC 4
3
21
SDLC
Requirement Initiation
Analysis & Specification
Risks
Delivery User Acceptance (UAT)
Deployment
Maintenance and Support
Operations
4
4 }
( '
Planning Scope, Time and Cost
Prioritization
Development Design / Prototype
Implementation
Integration
Testing
Documentation
Release
Software-Development-Life-Cycle
![Page 5: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/5.jpg)
5Agile process: Scrum
SPRINT REVIEW /RETROSPECTIVE
2-4 Weeks
1 2 3
4
![Page 6: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/6.jpg)
6
“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
The Agile Manifesto, First Principle
![Page 7: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/7.jpg)
WHY CONTINUOUS
TESTING?The metaphor
![Page 8: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/8.jpg)
8Software-Development-Life-Cycle Metaphor
1.Jack is a happy programmer, he has just done his first commit
2. After a few days, he receives the test report from QA Team reproving his changes and pointing a new BUG X in production.
3.Jack is sad now, but he fixes the bug and submit his changes again.
4. Now it is the week of a new release and Jack receives other report confirming the BUG X is fixed
5. But the report also contains new BUGS: A, B, C.
![Page 9: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/9.jpg)
9
6.Jack tries to figure out if his changes caused the other bugs
7.But Jack is not alone, he tries to figure out if someone else changes caused the other bugs
8.Jack tries a new solution to get done and re-tested in all environments before the deadline
9.Poor, poor, Jack… it is too late… the deadline has come.
Software-Development-Life-Cycle Metaphor
![Page 10: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/10.jpg)
10
What does Jack do?
Software-Development-Life-Cycle Metaphor
![Page 11: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/11.jpg)
11
Let’s think for a moment…
What are bugs? What do they eat? Where do they live? How to avoid new bugs?
What if bugs were living entities…
What are they thinking?
Bugs….
![Page 12: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/12.jpg)
12
"I WILL BE BACK"(The Bug)
)
![Page 13: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/13.jpg)
13
TRUST ME, I CAN DO IT.. ( Developer in a good day)
![Page 14: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/14.jpg)
14
• To control side effects • To avoid software defect snowball• To efficiently support multiple platforms,
environments and release versions • To get fast results in a short time cycle • To anticipate risks• To create an agile, reliable and repeatable process• Release in hours not months
Use Continuous Testing…
![Page 15: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/15.jpg)
CREATING YOUR FIRST PROJECT
IN MINUTES
Hands-On
![Page 16: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/16.jpg)
16Hands-On
• Let’s create our first project for continuous testing using an amazing and totally free web hosting platform with continuous integration support:
Let’s start from the test project template:
https://gitlab.com/fabricioepa/gitlab-ci-quickstart
gitlab.com
![Page 17: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/17.jpg)
17Hands-On•You can learn how to implement your own test platform
in the training:
https://fabricioepa.wordpress.com/2015/02/02/quickstart-for-continuous-integration/
•For more advanced continuous integration practices see the link:
https://fabricioepa.wordpress.com/2015/02/03/continuous-integration-practices-to-improve-the-software-quality/
![Page 18: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/18.jpg)
HOW IT WORKS
Overview
![Page 19: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/19.jpg)
19Overview
• Test Environment• Test Data Management • Automatic Deployment • Multiple environment configuration profiles •Virtualization (production-like) •Dynamic Infra-structure
•Test Automation • Orchestration
*Continuos Testing
![Page 20: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/20.jpg)
20Related Topics
• Continuous Integration (CI) • Continuous Delivery • Continuous Deployment • See 3 high levels of continuous testing adoption
*Continuos Testing
![Page 21: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/21.jpg)
Design development pipelines 21
Commit Stage Unit Tests
Code Analysis
Code Review
'
Triggers
/branches/product-1.0
Development Unit Tests
Code Analysis
Development pipeline
Build Package
Sanity Tests
Verify Metrics
Publishing Test Report
Unit Test Coverage
Project Metrics
Release Package
Fast Builds pattern ,
Artifact Repository$
01
![Page 22: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/22.jpg)
22
Setup Environment
Infrastructure & Data
QA automation pipeline
Deploy Install packages
Automated Test Plan Integration,
Functional, Regression,
Performance, etc.
Publishing Test Report
Test Coverage
Performance Benchmark
○ Push-buttonsScheduled buildsRepository changes{
' /branches/product-1.0
Triggers
Design QA pipelines
Self-service " + Nightly-builds {
Artifact Repository$
02
![Page 23: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/23.jpg)
23
Setup Environment
Infrastructure
Data
Testing Manual
Exploratory
○
Deploy Install packages
Health check
○
. QA Team
UAT
Self-service " deployments
Design delivery and deployment pipelines
Delivery / Deployment pipelines
User Acceptance Demonstrations
Consumer Preview
Beta testing
GA releases
03
Artifact Repository$
![Page 24: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/24.jpg)
AGILE TEST PROCESS
The real deal
![Page 25: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/25.jpg)
25Main Testing Stages
Planning Scope, Time, Cost
Process guideline
Specification Test Scenarios
Test Cases
Prioritization and
Classification
Execution Manual
Automatic
Closure Test Report
Bug Report
Qualification
Documentation
1 2 3 4
![Page 26: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/26.jpg)
26Test plan
A good test plan should answer at least 3 main questions:
• What? • How? • When?
01
![Page 27: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/27.jpg)
s27Test plan: What?
Guideline for macro activities•Test execution
• Defect detection (Bugs) • Enhancement areas
)
01
![Page 28: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/28.jpg)
• Strategies and methodology• Environment and tools
28Test plan: How?01
![Page 29: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/29.jpg)
• Traceability
29Test plan: How?01
![Page 30: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/30.jpg)
30
Schedule
Test plan: When?
/" 0Team and Resources
Test Scope and Risks
4
01
![Page 31: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/31.jpg)
31
•Black-box / White-box •Unit / Component /
Integration •Smoke / Sanity •Acceptance / Functional •Regression •System / End-To-end •Exploratory
•Load / Stress / Performance
•Parallel / Concurrent •Usability •Alpha / Beta •Compatibility •Destructive / Recovery •Internationalization /
Localization
Choose the right test techniques01
![Page 32: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/32.jpg)
32Test specification
• Requirements study •Test Scenarios and Estimative of Test Cases •Test Case Creation, Review and Maintenance •Prioritization and Classification
•Automatic vs. Manual •Others: Acceptance, Functional, Performance,
etc. •This information will be used for creating
execution test plans.
02
![Page 33: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/33.jpg)
33Executing test plans
•Manual execution •Automatic execution (by QA test pipeline) •Whatever the way you take, you must choose a
strategy to compute all test results in the final analysis
•To use a tool like TestLink may help you to manage all tests.
03
![Page 34: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/34.jpg)
34Closure
•Reports • Test Execution Status • Defects by Priority • Requirements Test Coverage • Other quality aspects:
•Performance, accuracy, security issues, etc.
04
![Page 35: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/35.jpg)
35Closure
•Final report and Qualification: •Approval Status of Deliverables
04
1
X1
Feature A
Feature B
Feature C
L Partial release approval
![Page 36: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/36.jpg)
The real value
EFFECTIVE CONTINUOUS
TESTING
![Page 37: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/37.jpg)
• Let’s remember the agile principle:
37Continuous Quality Improvement
“…continuous delivery of valuable software.” The Agile Manifesto, First Principle
• Testing != Software Quality Assurance • You must understand how to use tests to enhance
the process, reduce rework, increases the quality and business value of the software product.
• This is Continuous Improvement, one of the most important QA practices.
![Page 38: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/38.jpg)
38Continuous Improvement in 4 Steps
Plan Targets, Methodology,
Process, etc…
Do Process execution
Verification / Study Measure Results
Study evidences and deviations
from original plan
Action Diagnostic
Corrective and Preventive Action Points
Improvement Areas
![Page 39: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/39.jpg)
Notes: Continuous Improvement and Agile
• Retrospectives • Learned Lessons • Action Points • Use the PDCA Cycle 2
![Page 40: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/40.jpg)
40Effective Continuous Testing and Agile
•Objective •For each sprint the deliverable must potentially
be production-ready •Quality metrics and daily feedback
•Build status and reports •Statistics and Trends
•Automatic Tests report •Code violations •Code coverage •Project metrics
•Dev&QA teams must understand and maintain the quality metrics
![Page 41: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/41.jpg)
3 Displays + Dashboards
% Email notifications
Continuous Monitoring
Effective Continuous Testing and Agile
![Page 42: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/42.jpg)
42
•Balance test levels according to the project needs on each sprint
Effective Continuous Testing and Agile
Unit Test
Integration Test
System Test
Acceptance Test
Test individual component
Test integrated system
Test component groups
Test final system stack
V-model
![Page 43: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/43.jpg)
43
•Continuously improve your automated testsEffective Continuous Testing and Agile
Unit
Integration
E2E
• Recommended test distribution pyramid
Cheaper automation
Complex Automation
UI / Manual / Exploratory Tests
Reference: [1] http://googletesting.blogspot.com.br/2015/04/just-say-no-to-more-end-to-end-tests.html
![Page 44: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/44.jpg)
44
•Unrelated activities (to the backlog) •Implementing automation scripts •Status monitoring •Multiple environment maintenance: Testing,
Staging, Production… •Sprint Retrospective Tasks
•Use CI feedback to continuous improve the process and the product quality
•Diagnosing pitfalls and registering sprint action points
Continuous Testing and Agile Planning
![Page 45: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/45.jpg)
45Business Value
•The cost of adoption has a fast return on investment
•Rework reduction •Increase team productivity. •Release in days not months •Reduce at about 50% of development, testing
and deployment costs.
![Page 46: Continuous testing in agile projects 2015](https://reader034.vdocuments.site/reader034/viewer/2022042723/587097851a28ab412b8b6c29/html5/thumbnails/46.jpg)
!"
#
$
%
&'
$./run
{
Criado por Fabrício Epaminondas ©2015
ContactFABRICIO EPAMINONDAS 4 linkedin.com/in/fabricioepa
5 fabricioepa.wordpress.com
6 fabricioepa
7 @fabricioepa