property of progressive insurance & casualty company 1 agile test automation created and...
TRANSCRIPT
Property of Progressive Insurance & Casualty Company
1
Agile Test Automation
Created and Modified By:
Nishant AwasthiJanuary 13, 2009
Property of Progressive Insurance & Casualty Company
2
Agile Test Automation
Treat testware as software Application of agile development
principles to the test automation project.Requirements modeling.Automation architecture modeling.Acceptance testing of automated
scripts.Time Boxed iterations with frequent
delivery of testware.
Property of Progressive Insurance & Casualty Company
3
Agile Automation Process… contd.. Continuous integration of testware
Running tests with the latest development build
Continuous integrated testingExecute tests daily in the Development and
QA environment Provide continuous feedback
Lock-step with the development and the QA team for iterative releases of testware
Property of Progressive Insurance & Casualty Company
4
QA Cycle
The Automation Cycle
Development Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
Release
To Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
Property of Progressive Insurance & Casualty Company
5
The Automation Cycle
Development Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
Release
To Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
Development IPM
Development IterationPlanning Meeting
for the current iteration
Property of Progressive Insurance & Casualty Company
6
Development Cycle
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
Property of Progressive Insurance & Casualty Company
7
Development Cycle
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
Property of Progressive Insurance & Casualty Company
8
Automation CeremoniesName Frequency Agenda
Automation Planning Meeting (APM)
Every other weekFollowing DevelopmentIteration planningmeeting.
1) Review automation progress 2) Reprioritize any existing test cases as needed3) Review test cases prioritized for the current iteration
a. Overview of functionalityb. Determine automatabilityc. Determine dependencies (script modification, new
development etc.) d. Change Automation Status as needede. Assign automation developer to analyze the test plan and
create a task in Automation Tracker.
Automation Releasemeeting (ARM)
Every other week Following Development iteration releases.
1) Determine velocity and burn down.2) Determine current workload.3) Synchronize scripts for release.
Automation Acceptance Ad Hoc, as and when required.
1) Synch up with QA analyst and get feedback on the behavior and functionality of the script.
2) Mark the script for elevation in the daily run of automated regression Testware.
Property of Progressive Insurance & Casualty Company
9
Development Cycle
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
APM
Automation Planning Meetingfor the current iteration
Property of Progressive Insurance & Casualty Company
10
Automation PrioritizationAttribute Description
Business Risk of Not Automating Test If one script will be testing a higher-risk feature, it should be assigned a higher priority.
Number of Planned Executions If a script will be executed more times, it should be assigned a higher priority.
Efficiency (Automated vs. Manual) If one script will eliminate more manual testing it should have a higher priority.
Volatility of Features Being Tested If one script is testing a more volatile feature, it should be assigned a lower priority, since it will require more revisions due to the volatility.
Dependency of Other Tests If a high priority task depends on the completion of the candidate task, then the candidate task should also be assigned a high priority.
Reuse Potential Scripts which can be reused should have higher priority than one-off scripts.
Open Defects Scripts which test an open defect should be prioritized according to the severity/priority of the defect.
Automatibility If a test cannot be automated it should receive a low priority (at best). If it will take significant effort to automate, especially in relation to the benefit to the automation, it should also receive a low priority.
Usability If a script tests usability as opposed to proper functioning of the application, it should receive a lower priority.
Property of Progressive Insurance & Casualty Company
11
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationDevelopment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
AutomationScript Review
RequirementsSynch
Property of Progressive Insurance & Casualty Company
12
Model based test script design Model : description of a system’s
behavior Simpler than the actual systemState modeling is a common
practice in embedded systems world Finite State Model : consists of a set
of states, a set of input events and the relations between them.
Generate Sequences of Test Actions from the FSM
Property of Progressive Insurance & Casualty Company
13
Model based test script …contd… Application behavior model
Exploratory modeling WalkthroughsPaired exploration
Capturing states of the application
When is this action possible?What is the outcome of this action?
Automated test action sequences
Property of Progressive Insurance & Casualty Company
14
Automation Steps1) Test Plan Analysis
Analyze and refactor the manual test plan. Group similar test conditions.
2) Script design and development Modularize your script based on the different states or state transitions your
application Keep a single flow either positive or negative in each action that you record. Return back to the initial state in your script. Record the flow you are automating. Try to add all the objects on a particular page from the active screen. Use smart identification or vary the object recognition properties if required. Playback your script Repeat these steps for the negative flow
3) Parameterization Use the data table of QTP for positive data parameterization. Repeat the parameterization for negative data.
4) Checkpoints Use custom and built in checkpoints
5) Script Elevation Get your script acceptance tested and reviewed Elevate it to Test Director
Property of Progressive Insurance & Casualty Company
15
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
AutomationAcceptance
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
RequirementsSynch
AutomationDevelopment
AutomationScript Review
AcceptanceTesting
Acceptance
Automation acceptancebetween QA and automationdeveloper
Property of Progressive Insurance & Casualty Company
16
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
RequirementsSynch
AutomationDevelopment
AutomationAcceptance
AutomationScript Review
ARM
Automation release meetingbetween automationdevelopers
Property of Progressive Insurance & Casualty Company
17
The Automation Cycle
Unit Test andCode
EmbeddedTesting
AcceptanceTesting
AutomationTracker
Test PlanCreation
Story CardAssignment
AutomationPrioritizationand Analysis
AutomationAssignment
SourceControl
Build Proc
AutomationRelease
QA EnvDev Env
TestDirectorLab Machines
SystemTesting
Nightly and Ad Hoc
Nightly and Ad HocCheck-in
Build
ReleaseTo Prod
Manual Testing
Notes DB
RequirementsSynch
AutomationDevelopment
AutomationScript Review
AutomationAcceptance
Property of Progressive Insurance & Casualty Company
18
Roles and ResponsibilitiesQA Analyst Automation Developer Application Developer
1)Mark automation status pending after the test plan is created.2)Prioritize the test plan with respect to need for automation 3)Review and walkthrough the prioritized test plan to the automation developer.4)Notify the automation developer working on the test plan if there are any changes.5)Provide feedback to the automation developer after the automation status is “Ready for Release”
1)Pick up test plans that says “Creation complete” prioritized and assigned. 2)After picking up the test plan change Automation Status to In Process and Update Automation Comments field with name/time stamp.3)Estimate and define task(s) in Automation tracker (task planning system)4)Create automation script. 5)Synch up with QA analyst and get feedback on the behavior and functionality of the script after developing the script.6)Mark the test plan as ready for release after getting it reviewed.7)Release the test script into the daily run of “Automated Regression Testware” and mark automation status as completed.8)Facilitate automation ceremonies.
1)Keep automation team in loop while making UI changes.2)Help automation team with controls and technical details of the app as and when required.