in-process automation - hcl technologies · in-process automation – case study 8 ... hcl has...

WHITEPAPER DEcEmbER 2014

In-Process Automation

www.hcltech.com

Author: Sajitha RadhakrishnanSL CORE APPS-TESTING-TESTING-FT-BFSI

HCL Technologies Ltd., Bangalore, India

[email protected]

businessassurance& testing

IN-PROCESS AUTOMATION | DECEMbER 2014

© 2014, HcL TEcHNOLOGIES. REPRODUcTION PROHIbITED. THIS DOcUmENT IS PROTEcTED UNDER cOPYRIGHT bY THE AUTHOR, ALL RIGHTS RESERVED. 2

TABLE OF CONTENTS

INTRODUCTION 3

IN-PROCESS AUTOMATION – THE CONCEPT 4STANDARD REGRESSION AUTOMATION APPROACH

IN-PROCESS AUTOMATION APPROACH

IN-PROCESS AUTOMATION LIFE CYCLE

IN-PROCESS AUTOMATION – ADVANTAGES

CHALLENGES FACED

IN-PROCESS AUTOMATION – CRITICAL SUCCESS FACTORS

HCL’S IN-PROCESS AUTOMATION TOOL 7

IN-PROCESS AUTOMATION – CASE STUDY 8

CONCLUSION 9

ACKNOWLEDGEMENT 9

BIOGRAPHY 9

ABOUT HCL 10



INTRODUCTIONThe World Quality Report, published yearly by Capgemini, Sogeti and HP, is recognized as the IT industry’s largest research study. This report surveys the current quality and testing practices used across the globe as well as the emerging trends that may affect the future of testing.

Some of the key findings of the World Quality Report 2014-15 are:

y Quality Assurance functions are becoming structurally more mature –number of organizations with a fully-functional Testing Center Of Excellence increased from 4% in 2011 to 26% in 2014

y Organizations continue to increase the proportion of their IT budgets for testing – from 18% in 2012 to 26% in 2014. This is expected to rise to 29% by 2017

y The share of testing budget for transformational projects was 41% in 2012, which increased to 46% in 2013

y QA teams are still engaged too late in the application development life cycle, which contributes to the increase of testing’s share within IT budgets to manage operational and quality inefficiencies

All the findings from the report point to the fact that while testing as a function is growing, the IT industry is concentrating on improving the time to market and reducing the QA budget. The market is increased test automation and automating early in the application development process by using testing services.

The report also mentions that 40% of the global QA budget was estimated to be spent on functional testing, where only 20% was spent on regression testing and the remaining 20% on other testing activities. In traditional automation projects, the testing team has focused on automating only the regression tests. With the IT industry focusing on cost optimization, it has become imperative to focus on functional test automation. Even if we succeed to achieve 20% reduction in functional test execution effort, we could save millions of dollars.

The In-Process Automation approach proposes a shift left paradigm to the Software Development Lifecycle by automating the functional test cases instead of regression tests. This helps target 60% of our testing spend, rather than the traditional 20%. The automation development process runs parallel to the application development phase and the scripts get executed during the first execution cycle in the System Integration Testing (SIT) phase to reduce the testing cycle time.



IN-PROCESS AUTOMATION – THE CONCEPTTo understand the In-Process Automation approach better, let us look at the standard regression automation approach.

STANDARD REGRESSION AUTOMATION APPROACH

UAT Release & Maintenance Analysis Coding CIT SIT UATDesign

1 2 1 2 1 2 3 4 5 6 1 2 31 2 31 2 1 2WEEK 1 2

Previous Release Current Release

Test case preparation from manual team

1

Identification of test cases for Regression testing from the manual test cases

2

Scripting of regression Scenarios for previous release

3

Tweaking should be done for the scripts before execution if there are any changes from the previous build to the current build

Execution of Automation Regression pack

4

Cycle should repeat for CIT, SIT and UAT

The above diagram describes the traditional automation approach of automating the regression test cases.

Once the functional testing of an application is completed and signed off for a release, the manual team identifies the test cases to be added to the regression suite from the functional test suite. These test cases are then handed over to the automation team that conducts the feasibility and automates the regression suite during the development phase of the next release. This automation regression pack is then used for regression testing in the subsequent testing phases of the release. The automation is termed Lagging Release Automation. Now let us see how the In-Process Automation approach is implemented.

IN-PROCESS AUTOMATION APPROACH

Analysis Design Coding CIT SIT UAT Release & Maintenance

1 2 1 2 1 2 3 4 5 6 1 2 3 1 2 3 1 2 1 2

Identification of functionalities for automation

Framework definition

Feasibility study on Application

Scripting of 1st module

Scripting for nth module

Dry run of (n-1)th

module Scripts

Execution of entire Automation pack



Execution of Automation Regression pack

1

2

3

4

4

5

6 7

8

9

Entry criteria Suspension criteria Exit criteria

WEEK

Cycle should repeat for all the modules

In this approach, the automation development life cycle runs parallel to the application development life cycle. During the analysis and the design phase, the functionalities that need to be automated are identified and the automation framework is defined



and developed. If an existing framework is available, any necessary modifications are also completed. Once the application goes into the coding phase, the feasibility study of the application is conducted and the scripting begins. It continues throughout the coding phase and once the application reaches the Component Integration Testing phase, the first time execution of the automation pack is conducted. The necessary modifications are done with respect to the application and then the automation pack utilized for the subsequent phases of the life cycle. The same automation pack can then be utilized as the regression pack, providing maximum testing during every application release.

IN-PROCESS AUTOMATION LIFE CYCLE

To make this approach a success, a set of activities recommended to be performed during the course of application development life cycle. These are:

1. Understanding the business flow during the requirements phasea. Static testing and analysis of the Business Requirements Document (BRD),

Functional Specifications Document (FSD), and technical specifications

b. Understanding the functionality and deciding on the modules to be automated

c. Creating a common test template to be used for automation

Pre-conditions for the above mentioned activities to be conducted during this phase are:

a. Signed-off technical and functional specification documents

2. Develop/ modify the automation framework during the design phasea. Analyzing the system design

b. Creating the test scenarios

c. Designing the framework

d. Identifying the functional flow and object details

Pre-conditions:

a. Module prioritization information

b. Design documents and the application flow with the object names

c. An application prototype if available

3. Develop the automation scripts during the application development phasea. Use of programming approaches like descriptive programming to complete the

development of automation framework

b. Automation scripting with the object details and design flow available from the earlier phase

Pre-conditions:

a. Stable test environment

b. Software for the agreed modules

c. Application queue and database details, if applicable



4. Perform a dry run of the automated scripts during the unit testing/ Component Integration Testing (CIT) phase.a. Executing the script on developed modules, as and when the automation

scripting completes

b. Preparing the test data for the modules under development

Pre-conditions:


5. Perform automated functional testing during the SIT phasea. Reworking on the framework and the test scripts for any gaps identified in

correspondence to the application under test

b. Complete the functional automation testing and share the results

c. Analyze and decide on the automation scripts that can be used by User Acceptance Testing (UAT) team

Pre-conditions:


b. Faster defect turnaround

6. Maintenance and execution of the automation scripts in the subsequent phases and across releasesa. Select the tests to be reused for regression

b. Perform the regression tests for subsequent releases

Pre-conditions:


b. Faster defect turnaround

IN-PROCESS AUTOMATION – ADVANTAGES

1. Achieve early test automation, resulting in reduced manual testing effort

2. Increased test coverage caused by high volume of testing through automation

3. Reduced time to market/ cycle time caused by automation

4. Reusability of automated test scripts across CIT, SIT, UAT, and regression

5. Early defect detection during functional testing

6. Increased regression scope with functional test automation

CHALLENGES FACED

1. High script maintenance effort for In-Process test automation

2. Frequent design changes impact the automation scripts

3. Automation development based on design document and scripts will be validated only once the application development is completed



IN-PROCESS AUTOMATION – CRITICAL SUCCESS FACTORS

y Stable application environment during the coding and subsequent phases – With the automation team working on development environments, stable application environment and scheduled downtimes are important

y Historically low volume of defects during CIT –Helps in less rework and maintenance

y Stability index to remain high – There should be minimal or zero changes to the functional specifications document during the coding phase to ensure less maintenance of the developed scripts

y The automation team should coordinate with the development team to prioritize the modules coding based on mutually agreed automation plan

y The development team should be able to provide faster resolution of all defects identified, even during the development window. This ensures faster coding of automation scripts with very less downtime

y A common test template should be used across testing teams for test scenario preparation for consistency. This test template should be designed taking into consideration the requirements and expectation of the automation team

y All specification documents should be delivered to the automation team as per plan indicating the object, interfaces, and navigation flow information

HCL’S IN-PROCESS AUTOMATION TOOLHCL has developed a ready-made tool capable for evaluating the In-Process Automation feasibility and applicability for the application under test based on a comprehensive set of inputs.

Some of the inputs include:

1. Application and project methodology

2. Program release information

3. Historic defect and turnaround information

4. Documentation for the application and release

5. Environment details

6. Test scenario/ case information

7. Execution and productivity information

HCL’s tool provides the following outputs:

1. A feasibility score of the application indicating whether the solution is technically viable

2. An applicability score which indicates the Return on Investment (ROI) on the efforts spent



3. Recommendations to improve the feasibility and applicability scores

An Applicability Index less than -20% (translates to 20% additional effort compared to traditional approach) is typically ‘Not Recommended’ for In-Process Automation. The In-Process Automation is applicable even if there is a 20% additional effort - as there could still be a reduction in execution cycle time. An applicability index less than 10% is considered as “Feasible”. An applicability index greater than 10% is “Strongly Recommended” for In-Process Automation.

A Feasibility Index less than 30% is typically “Not Recommended” for In-Process Automation. A Feasibility Index between 31% and 70% is categorized as “Feasible” for In-Process Automation, though some level of process tweaking would be required (provided in Recommendation section). A Feasibility Index greater than 70% is a strong contender for “High Feasibility” for In-Process Automation.

IN-PROCESS AUTOMATION – CASE STUDYThe In-Process Automation approach was implemented for one of HCL’s investment banking clients for a platinum project. HCL had proposed to automate at least 50% of the functional test cases during the development phase. The grid below indicates the expected effort saved for In-Process Automation (without manual execution) against the effort spent on automating the test cases, which would have been executed at least once manually.

Activities Case 1 Case 2

Number of test scenarios (Automatable) 1000 1000

Number of test cases (Automatable) 3000 3000

Test scenario preparation effort (Person days - Assuming productivity of 20 TS per day) 50 50

Test cases preparation effort (Person days - Assuming productivity of 40 TC per day) 218 109

Test data preparation effort (Person days - Assuming 20% of test case creation effort) 44 44

Test cases planned to be automated 0 1500

Automation scripting effort (Person days - Assuming productivity of 3 TS per day) 0 167

Total preparation effort - Manual (Person days) 312 203

Total preparation effort - Automation (Person days) 0 167

Total preparation effort (Person days) 312 370

Manual execution effort (Person days - Assuming productivity of 25 TC per day and 4 cycles) 480 240

Automation execution effort (Person days - Assuming productivity of 200 TC per day and 4 cycles) 0 30

Total execution effort (Person days) 480 270

Total effort (Person days) 792 640

Percentage Effort save between case 1 and case 2 24%



CONCLUSIONThe In-Process Automation approach helps to start the automation process early and get the benefits of automation throughout the testing life cycle. A transparent development life cycle and coordination between the testing and development teams helps this approach succeed. As the automation test execution is faster, defects can be detected early in the testing cycle, ensuring more defect fixing time for the development teams. The development and testing teams need to deploy a disciplined approach since the automation cycle runs parallel with the development life cycle. The manual testing team also gets an opportunity to conduct random/ ad hoc testing to discover niche defects.

In today’s industry where decreasing the time to market is the motto for any application, an approach like In-Process Automation should be implemented to facilitate the testing process and the team.

ACKNOWLEDGEMENTThe author gratefully acknowledges the contributions of Raam Kumar, Neha Kotak, and the members of her project team for their work on the original version of this document.

BIOGRAPHYSajitha Radhakrishnan is an ISTQB Certified Test Manager with 10 years of experience in Manual and Automation Testing in the Finance Domain. She has expertise in different types of testing and testing methodologies for multi-tier applications and strong hands-on knowledge of several process analysis approaches/ tools. She has extensive experience in test planning, estimation, status tracking and reporting, metrics collection, and preparation.

Sajitha currently manages implementation of the In-Process Automation approach for one of the platinum projects of an HCL client. This is a messaging platform migration project where the application under test interfaces with 200+ back end applications. The automation testing includes SWIFT Network Validation testing for around 216 message types, message comparison using service virtualization, and functional testing using Quick Test Professional and Green Hat.

ABOUT HCL

About HCL Technologies

HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on ‘transformational outsourcing’, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of ‘Employees First, Customers Second’ which empowers its 95,000 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.5 billion, for the Financial Year ended as on 30th September 2014 (on LTM basis). For more information, please visit www.hcltech.com

About HCL Enterprise

HCL is a $6.7 billion leading global technology and IT enterprise comprising two companies listed in India – HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India’s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 100,000 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit www.hcl.com

Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 100,000 Ideapreneurs are in a Relationship Beyond the Contract™ with 500 customers in 31 countries. How can I help you?

in-process automation - hcl technologies · in-process automation – case study 8 ... hcl has...

Documents