qa effort estimation by estimator metrics @ stc 2009
DESCRIPTION
This presentation enlightens the novel approach for estimation of QA time and resources. Numerous approaches e.g. Function Points Analysis, Feature points, Use Case points or Lines of Code (LOC) available in market specifically for project estimation but QA time and effort estimation generally done through past projects experience or simply through wild guess.Some time it gives correct estimation but the majority of time it confers lack of time for QA activities.This paper is an attempt to offer tiny step of concrete approach for QA estimation using priority of influence elements and impact factor.TRANSCRIPT
Estimator Metrics A Novel approach for QA estimation
Presenter’s
AMIT BHARDWAJLead, Quality EngineeringTekMindz India Pvt. Ltd., Noida
Agenda
Traditional QA Estimation [ Usually “Guess” Based]
Probable Hurdles with traditional Approach
“Estimator”, A novel approach for QA estimation
Case Study : Implementation on “e-Insurance”, our on going
project.
Traditional QA Estimation [ “Guess” Based ]
Based upon past experience & expertise in technology.
Based upon development cycle (60%-40%)
Experience from repeated Production Customization activities
QA estimation based upon the above fact is fine! If and only if it is a short term, less complex and low priority project.
A long term project with high priorities and complexities, shouldn't be estimated based on the traditional guess based approaches.
Estimation will be done more then once
(Inadequate estimation)
Risk of Client back out
(Missing deadlines)
Financial loss of organizations
Multiple time estimation will affect Entire Project and
Business Plan
Probable Hurdles with the Traditional Approaches
Constituent of
Test Estimation
Calculation of Impact Factor
Assumption for Deg. Of QA activities, Risk Priorities & Resultant
factor
Identification of
QA activities
Preface of
Estimator Metrics
“Estimator Metrics” A novel approach for QA estimation
It’s not rocket science!! Just precisely learn the art of using ESTIMATOR !!Following are the steps to achieve reliable effort estimation:
1. Constituent of Test Estimation
2. Calculation Mechanism for Impact Factor
3. Assumption for Degree of QA activities, Risk Priorities & Resultant Factor
4. Identification of QA activities
5. Preface of Estimator Metrics
Constituent of QA
Estimation
Approach
Scope
Prerequisite Info
H/W & S/W availability
Man Power Efficiency
Risk & Mitigation
Plan
Client’s zeal for Quality
The seven constituent of estimation has been identified, those having extreme tie with estimation activities. It is recommended to review and keep in mind! Before moving ahead!
Approach Test process, Test design techniques, Entry & Exit criteria and Test types (RBT/SBT)
ScopeFunctional, Performance, Migration and Localization
Prerequisite InfoBRD /SRS/ Use Cases availability.
H/W & S/W availability (Infrastructure Establishment)Servers, licenses and equipments etc
Man Power EfficiencyDomain expertise, Anticipated experience
Risk & Mitigation PlanInfrastructure Investment, Documentation, Frequent Change Requests, manpower backup
Client’s zeal for QualityWhen to stop testing?? , Will agreed upon Adequate time & resources?
Step 1: Constituent of QA Estimation
Impact Element
Project
Priority
Major
High 8
Medium 4
Low 2
Minor
High 4
Medium 2
Low 1
Onsite Support
Complexity
of work
Technology
QA Team
Classification of
Impact element
Degree of
Impact element
Risk Priority of
Impact elementResultant Factor
Step 2: Calculation Mechanism for Impact factor
IMPACT
FACTOR
A. Classification of Impact Element:
Classification of Impact element
Project Priority
Onsite Support
QA TeamTechnology
Complexityof work
Step 2: Calculation Mechanism for ..contd. Classification of Impact element
2. Complexity of Work : (Based on I/O Functions)Banking/ Insurance /Mobile, and third party transaction involvement OR simple an employee data mgmt , No third party transaction
3. Technology : Proven (Completely) OR New
4. Onsite Support (elective) : Available OR Not
5. QA Team :Has enough professional experience and domain as well as technical expertise ORhaving intermediate expertise.
1. Project Priority :
Business impact. Security level OR simple e-Learning , Information search applications
All right!! Now we have defined the impact elements, furthermore we need to identify the valid degree for all the impact elements. (Prepare Standard Check List for the same)
B. Degree of Impact Element
1.Project Priority: Critical Business prospect of application. Highly Secure application MAJORSimple e-Learning , Info Search App. MINOR
2.Complexity of Work: Banking/ Insurance /Mobile, and third party transaction involvement MAJORSimple employee data management , No third party transaction involved MINOR
Major OR
Minor
Project Priority
Onsite Support
QA Team
Technology
Complexity of Work
Step 2: Calculation Mechanism for ..contd.
Degree of Impact Element
Major & Minor
Note : Major implies the soaring effort while Minor implies fewer effort .
C. Risk Priority of Impact element
Now, we need to identify and categorize the risk involved for impact elements.
High Risk: (Probability 70-100 %)Would require high effort / time to handle the impact element.
Medium Risk: (Probability 40-70 %)
Would require average effort / time to handle the impact element.
Low Risk: (Probability 0-40 %)
Discipline within the project is familiar with impact element.
Step 2: Calculation Mechanism…contd.
Risk Priority of Impact Element
Degree of Impact element
Major
High
Medium
Low
Minor
High
Medium
Low
Risk Definition
D. Resultant Factor representation (A numeric representation for Risk Priority)
We opted to start the same with “1” (To avoid decimal calculation conflicts ). It is entirely based upon iterative approach started from 1 to achieve the reliable test estimation. (Fortunately, we got satisfactory outcome while started with “1” )
Low resultant factor for Minor degree as ‘1’, which is then subsequently increasing for the respective combinations.
Step 2: Calculation Mechanism ..contd.
Resultant Factor representation
E. Metrics based on Degree of Impact element, Risk Priority & Resultant Factor
Below table demonstrates standard resultant factor’s for all the impact element corresponding to measured (for hypothetical project) degree of impact elements
Step 2: Calculation Mechanism …contd.
Metrics based on Degree of Impact element, Risk Priority & Resultant Factor
It’s simple, you need to add applicable resultant factor values and divide the sum by total number of Impact element i.e. 5.
Step 2: Calculation Mechanism for Impact factor
F. Calculation of Impact Factor
***Calculation for Deg. of Impact element, Risk Priorities and Resultant Factor is based upon hypothetical project.
Impact factor is calculated as (= 23 / 5) = 4.60
Impact factor = [ ∑ Impact Factors / No. impact element ]
To calculate Test time Estimation, we need to define degree of QA activities and corresponding risk priority & resultant factors.
Step 3: Assumption for Degree of QA activities, Risk Priorities & Resultant Factor
"Quality" refers to all the good things that we would like to see in our product. We build the product by assuring that its quality was kept in mind by performing various activities at different levels. Core QA activities can be defined as below:
• High Level Requirements / SRS / Use Case documents review• Test Case Designing• Test Case Execution / Updation • Functional QA (Test Case execution, Regression)
More QA activities can be defined as per requirement. Available features and functionalities can be broken down for respective QA activities.
Step 4: Identification of QA activities
Following are the available feature / functionality in the application (assume):
F1 F2 F3
Features will be broken down for the respective QA activates.
Step 5: Preface of Estimator Metrics
Case Study: Implementation of Estimator on actual QA project
Initial Guess Based Estimation
Case Study: Initial Guess Based Estimation Submitted to Client
Due to earlier Inadequate Estimation, we
forced to buy extra time from client to complete the tasks.
First Estimation time = 87 daysforced to buy extra time = 30 daysTotal time taken to finish the tasks = 117
QA time estimation for the QA activities using
Estimator = 122
Estimator approach seems much better then earlier one !!!
