testing in agile development
DESCRIPTION
Presented in BSPIN Conference (http://bspin.org/conference2014/) on "Succeeding in SMAC World". Had great interactions and glad to see great interest on Agile Testing concepts with Participants.TRANSCRIPT
![Page 1: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/1.jpg)
Hariprakash Agrawal [email protected]
8th BSPIN Conference on 18-19, September 2014
http://testoptimizer.com
Testing in Agile Development
![Page 2: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/2.jpg)
Services Experience
About Me
Serving as CEO at OpCord – Software Development, Test
Automation, Agile Consulting, Products (RideAlly, TestOptimizer)
Agile / Kanban Coach & Trainer (CSM) – Agile Transformations
Email: [email protected]
2
Enterprise Agile Coaching & Transformation
Building Agile Culture and values
Training Scrum, Kanban, XP
Implementing SAFe Framework
Tools expert: Rally
An Entrepreneur and Agile Coach
http://testoptimizer.com
![Page 3: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/3.jpg)
The Problem
http://testoptimizer.com 3
![Page 4: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/4.jpg)
Individuals and interactions over Processes and Tools
Working Product over Comprehensive Documentation
Customer Collaboration over Contract Negotiation
Responding to change over Following a plan
Agile Manifesto
That is, while there is value in the items on the right, we value the items on the left more.
Courtesy: www.agilemanifesto.org 4
![Page 5: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/5.jpg)
Agile Principles
5
Our highest priority is to satisfy the customer through early and continuous delivery
Welcome changing requirements, Agile processes harness change for the customer's competitive advantage.
Deliver working product frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working product is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity--the art of maximizing the amount of work not done--is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Courtesy: www.agilemanifesto.org
![Page 6: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/6.jpg)
Courtesy: http://smartbear.com/products/qa-tools/what-is-agile-testing/ 6
![Page 7: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/7.jpg)
Scrum Life Cycle!
7 http://testoptimizer.com
![Page 8: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/8.jpg)
Agile Testing with Scrum
Courtesy: https://www.imbus.de/veranstaltungen/trends-in-testing-2012/ 8
![Page 9: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/9.jpg)
Definition of Done
9
is usually a clear and concise list of requirements that a software increment must adhere to, for the team to call it complete
checklist of valuable activities required to produce software
primary reporting mechanism for team members
not static
an auditable checklist
DoD can be at various levels • for a feature (story or product backlog item) • for a sprint (collection of features developed within a sprint) • for a release (potentially shippable state
Courtesy: http://www.scrumalliance.org/community/articles/2008/september/what-is-definition-of-done-%28dod%29
![Page 10: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/10.jpg)
Sample Definition of Done
User Story Level
Code checked-in the integration branch and merged
Code review done and comments incorporated (Coding guidelines checked)
Automated Unit tests run
code coverage tools run Static analysis is at 100% pass rate
No open defects unless deferred
Product owner demo completed
Acceptance criteria met
10
Sprint or Release Level
Required user documentation updated
No open defects unless deferred
Test cases documented, automated and run over night
Functional testing is at 100% pass rate (Positive and Negative)
Usability guidelines incorporated User-facing story is reviewed by UE designers (where applicable)
Product owner demo completed
Acceptance criteria met
http://testoptimizer.com
![Page 11: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/11.jpg)
Extreme Programming!
User Stories
Release Planning
Sprint Planning
On Site Customer
Pair Programming
Test-first Design
Refactoring
Continuous Integration
Collective Ownership
Coding Standard
40 hour week
11 Courtesy: http://xprogramming.com/what-is-extreme-programming/
![Page 12: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/12.jpg)
Agile testing is a software testing practice that follows the principles of agile software development#
Courtesy: http://en.wikipedia.org/wiki/Agile_testing
– involves all members of a cross-functional agile team#– with special expertise contributed by testers#– to ensure delivering the business value desired at frequent intervals#– working at a sustainable pace #– specification by example is used to capture examples of desired and
undesired behavior and guide coding.#– not a separate phase, but an integral part of software development, along
with coding#– "whole-team" approach to "baking quality in" to the software product#– testing and coding are done incrementally and iteratively#– covers all types of testing#
12
![Page 13: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/13.jpg)
Traditional Vs. Agile Testing
Courtesy: http://www.agilecoachjournal.com/index.php/2014-01-28/testing-2/the-agile-testing-pyramid/ 13
![Page 14: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/14.jpg)
Principles of Agile Testing
Testing moves the project forward
Testing is NOT a Phase
Everybody Tests
Reduce feedback latency – Agile teams test early and often
Test represent expectations
Bugs don’t hang around – Keep code clean & Fix bugs fast
Reduce test documentation overhead
Tested is part of “Done”
From test last to test driven
Courtesy: http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf 14
![Page 15: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/15.jpg)
Practices of Agile Testing
Automated Unit/Integration Tests
Continuous Integration
Test Driven Development
Automated System Level Regression Tests
Acceptance Test Driven Development
Behavior Driven Development
Exploratory Testing
Courtesy: http://testobsessed.com/wp-content/uploads/2011/04/AgileTestingOverview.pdf 15
![Page 16: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/16.jpg)
7 deadly sins of Agile Testing
Courtesy: http://properosolutions.com/2011/03/the-seven-deadly-sins-of-agile-testing/
Water scrumming
Lack of TDD and CI
Testing is one Sprint behind
Coding
Unbalanced Testing
Quadrants Ignoring Test
Failures
Separate QA Team
Requirements and Tests are
Separate
16
![Page 17: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/17.jpg)
Exercise
Predict the quadrants
http://testoptimizer.com
Q2# Q3#
Q1# Q4##
Business Facing#Su
ppor
ting
the
Team
#Critique Product#
Technology Facing#
17
![Page 18: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/18.jpg)
Agile Testing Quadrants
Courtesy: http://swansegarconsulting.com/learn-agile/agile-roles-titles/quality-assurance-control/ 18
![Page 19: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/19.jpg)
Have we decided to automate tests?
http://testoptimizer.com 19
![Page 20: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/20.jpg)
Why Automate Tests?
http://testoptimizer.com 20
![Page 21: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/21.jpg)
Test Automation is the use of special software to control the execution of tests and the comparison of actual outcomes with predicted outcomes.
Courtesy: http://en.wikipedia.org/wiki/Test_automation
Code-driven testing The public (usually) interfaces to classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.
Graphical user interface testing Generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.
API driven testing Uses programming interface of the application to validate, the behaviour under test. Typically API driven testing bypasses application user interface altogether.
21
![Page 22: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/22.jpg)
Test Automation Life Cycle - 1
Courtesy: http://www.nordixsoft.com/AutomatedTesting/AutomationMethodlogy/ 22
![Page 23: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/23.jpg)
Test Automation Life Cycle - 2
Courtesy: http://geek4eva.com/2009/08/28/test-automation-and-roi/ 23
![Page 24: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/24.jpg)
Types of Automation Frameworks
Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/ 24
![Page 25: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/25.jpg)
Linear Framework
Verify the creation of a new account in gmail!#Steps are:##a) Open gmail.comb) Click on ‘Create Account’ c) Enter the details d) Verify the detailse) Create the account##
#1 'Open GMail#2 SystemUtil.Run "iexplore.exe", "http://www.gmail.com”#3 'Page Sync#4 Browser("Gmail").Page("Gmail").Sync#5 ‘Click on create account#6 Browser("Gmail").Page("Gmail").WebLink(“Create Account”).Click#7 ‘Enter the details#8 Browser("Gmail").Page("Google Accounts").WebEdit(“First Name”).Set “Swati”#9 Browser("Gmail").Page("Google Accounts").WebEdit(“Last Name”).Set “test”#10 ‘Fill in several other details#11 ‘Submit#12 Browser("Gmail").Page("Google
Accounts").WebButton(“Next Step”).click#
25 Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/
![Page 26: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/26.jpg)
Keywords Driven - 1
Function gotoGoogleAccount()#'Open Gmail#SystemUtil.Run "iexplore.exe", "http://www.gmail.com"#'Page Sync#Browser("Gmail").Page("Gmail").Sync#‘Click on create account#Browser("Gmail").Page("Gmail").WebLink(“Create Account”).Click#‘Enter the details#End Function#!
Function SubmitToCreate()#‘Submit#Browser("Gmail").Page("Google
Accounts").WebButton(“Next Step”).click#
End Function!!##Function EnterDetails()#Browser("Gmail").Page("Google
Accounts").WebEdit(“First Name”).Set “Swati”#
Browser("Gmail").Page("Google Accounts").WebEdit(“Last Name”).Set “test”#
‘Fill in several other details#End Function#
26 Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/
![Page 27: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/27.jpg)
Keywords Driven - 2
'Open GMail gotoGoogleAccount() ‘Enter the details EnterDetails() ‘Submit SubmitToCreate() It helps in achieving readability, modularity and if in case another program wants to use the login function, we can surely reuse it.
27 Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-17-test-automation-frameworks/
![Page 28: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/28.jpg)
Data Driven
Courtesy: http://www.softwaretestinghelp.com/qtp-tutorial-18-data-driven-and-hybrid-frameworks/
Browser("Gmail: Email from Google").Page("GoogleAccounts").WebEdit("FirstName").Set "swati"#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("LastName").Set "s"#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("GmailAddress").Set "test”########Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("FirstName").Set DataTable("G_First_Name", dtGlobalSheet)#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("LastName").Set DataTable("G_Last_Name", dtGlobalSheet)#Browser("Gmail: Email from Google").Page("Google Accounts").WebEdit("GmailAddress").Set DataTable("gmail_address", dtGlobalSheet) ###
28
![Page 29: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/29.jpg)
Exercise
Take any scenario and write an approach (or code) keeping Hybrid framework in mind
http://testoptimizer.com 29
![Page 30: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/30.jpg)
Distributed Test Automation
Courtesy: http://www.testandtry.com/2009/02/19/distributed-test-automation-infrastructure-plan-1/ 30
![Page 31: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/31.jpg)
Continuous Integration
31
Maintain a code repository
Automate the build
Make the build self-testing
Every commit (to baseline) should be built
Keep the build fast
Test in a clone of the production environment
Make it easy to get the latest deliverables
Everyone can see the results of the latest build
Run regression test in automated way over night
Automate deployment
http://testoptimizer.com
![Page 32: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/32.jpg)
Test Driven Development (TDD)
Courtesy: http://en.wikipedia.org/wiki/Test-driven_development 32
![Page 33: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/33.jpg)
http://testoptimizer.com 33
![Page 34: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/34.jpg)
What ATDD takes?
34
Executable requirements
Product Owner perspective
Continuous refactoring
Tests drive product design
Automated acceptance tests
Example: Selenium
http://testoptimizer.com
![Page 35: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/35.jpg)
Behaviour Driven Development (BDD)
Courtesy: http://blog.bughuntress.com/automated-testing/automated-testing-with-behavior-driven-testing 35
![Page 36: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/36.jpg)
Behaviour Driven Development
Courtesy: http://behaviordrivendevelopment.wikispaces.com/Overview 36
![Page 37: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/37.jpg)
BDD in Nutshell
Courtesy: http://www.hascode.com/2011/05/oh-jbehave-baby-behaviour-driven-development-using-jbehave/ 37
![Page 38: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/38.jpg)
Factors to consider for test automation tool
http://testoptimizer.com
Extensible & Customizable (Open APIs)
Runs with automated
build over night (batches)
Customizable Reporting
Easy Debugging & Logging
Email Notifications
Keywords & Data Driven Framework
Take Screenshots
for failed testcases
Multi Browser & OS
Support
Platform Independence
Supports Distributed Execution
Environment
38
![Page 39: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/39.jpg)
Test Automation Tools – Open Source
Courtesy: http://www.opensourcetesting.org
Opensourcetesting.org lists many open source testing tools
Functional 129 Performance 53 Test management 29 Unit testing Java 72, C/C++ 39, .Net 20, PHP 10
39
![Page 40: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/40.jpg)
Tools as per Quadrants
Courtesy: http://www.slideshare.net/AndrewDzynia/effective-testing-in-agile?qid=ec8c463b-4b6a-41ea-80c9-064b7b4d5bd1&v=qf1&b=&from_search=6
Story tests, Checklists, Mind Maps, Flow Diagrams#
#Automation Tools: BDD
(Fitnesse etc) and UI Tools (Selenium etc)#
Q2 Tools, Monitoring Tools#Data Generation tools#
Exploratory testing tools (Rapid Reported, Session Tester)#
Code Management: SVN, GIT#IDE: Eclipse, NetBeans#
Build/CI: Jenkins, Ant, Maven#UT: xUnit, Mocks#
Loadrunner, Silk Performer,#Jmeter#
Profilers##
Business Facing#Su
ppor
ting
the
Team
#Critique Product#
Technology Facing#
40
![Page 41: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/41.jpg)
More information about tools
Tool Type # #WHO # # #URL ##BDD # # #Cucumber #http://cukes.info/##GUI UT# # #Abbot # #http://abbot.sourceforge.net/doc/overview.shtml#GUI UT# # #SWTBot #http://www.eclipse.org/swtbot/#GUI UT# # #TestNG# #http://testng.org/doc/index.html##UT # # #EasyMock #http://www.easymock.org/#UT # # #Junit for Java #http://www.junit.org/##UT # # #Nunit for .NET #http://www.nunit.org/#UT # # #PyUnit for Python #http://pyunit.sourceforge.net/#UT # # #Ruby/Mock#https://www.ruby-toolbox.com/categories/mocking#UT # # #Perl and Ruby #http://test-unit.rubyforge.org/#
http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks ##
41 http://testoptimizer.com
![Page 42: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/42.jpg)
Recent Trends
Test Automation is possible directly from testcases in Excel Test Driven Development is possible without coding or any knowledge of programming
Simplified BDD
Anybody can write/modify Testcases (in plain English) based on user stories in Excel Anybody can run/execute them by clicking a button
Ease of Use
No dependency on specific skills or automation engineers. Even Management, Business analysts can change/run testcases Run testcases on any OS/Browser and its versions
No Dependency
Smaller Test Cycles Low License cost Only basic training needed, No learning curve
Cost
42 http://testoptimizer.com
![Page 43: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/43.jpg)
TestOptimizer helps you automate Functional and UI Test Cases without writing a single line of code
Business Analyst or Testers write testcases
as usual in Excel or spreadsheets
Testers also write test steps to enable
automation in Excel itself
Install TestOptimizer and store test cases
sheet in defined location
Just click on TestOptimizer icon to
execute testcases in automated way
Choose browsers on which you wish to run
testcases
Get screenshots for failed testcases and get results with pass/fail in
same Excel
Get test reports in xls or html format
Get email with consolidated test execution reports
That’s All !!! Automation done !!!
43 http://testoptimizer.com
![Page 44: Testing in Agile Development](https://reader034.vdocuments.site/reader034/viewer/2022042613/546e12f9b4af9f702c8b5686/html5/thumbnails/44.jpg)
OpCord Consultancy Services Private Limited, 2nd Floor, #417, Above People Showroom, 27th Main, Sector -1, HSR Layout, Bangalore – 560102
Voice: 91 – 9620887298, Email: [email protected], Web: http://opcord.com
Let’s Opcord
Let’s optimize with coordination