oow16 - testing oracle e-business suite best practices [con6713]

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

Testing Oracle E-Business Suite Best Practices Keeping Up with Change…

Confidential – Oracle Internal/Restricted/Highly Restricted

Presented with

Mazen Arar Senior Director, Oracle Sarvesh Tiwari Vice President, Genpact September 21, 2016


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted 3


Program Agenda

EBS Evolution

Testing Readiness

Recommendations & Tools

Customer Story

1

2

3

4

4 Confidential – Oracle Internal/Restricted/Highly Restricted


EBS Evolution Change is Constant

• Change happens every day

• In today’s business world, it is the only thing that is consistent

• Long-term plans may become outdated and possibly obsolete within months

• Business plans must be reviewed, updated, and revised constantly



EBS Evolution Oracle E-Business Suite in the Digital World

Cloud Social Mobile

IoT Big Data

Talent Mgmt

Revenue Mgmt

Recruiting

Innovation Mgt Commerce

Financial Reporting

Global Trade Mgmt

Enterprise Planning

Service

Transportation Mgmt

Sales

Marketing



Program Agenda

EBS Evolution

Testing Readiness


Customer Story

1

2

3

4



Testing Readiness Challenging Facts

• Continuous development calls for continuous testing

• With ongoing EBS innovation, all teams need to be aligned

• Different types of activities require different strategy for testing

• QA teams have to be pro-active not re-active

• Focusing on the present and planning for the future



Testing Readiness Ongoing EBS Innovation

• New features / Products

– EBS 12.1

– EBS 12.2

– Modern / Simplified UI

• Mobile

• In Memory Applications

• Big Data - Information Discovery

• Cloud

• IoT



Testing Readiness Strategy

10

Smartphone Applications

In Memory Applications

Information Discovery

Cloud

IoT


https://emeapressoffice.oracle.com/Media-Library/Oracle-SPARC-M6-32-be3.aspx


Program Agenda

EBS Evolution

Testing Readiness


Customer Story

1

2

3

4



Recommendations & Tools Where Do You Start?

12

Upgrades




Automate

Execute

Plan




• What is the task on hand

• Available resources

– Release Content Document • Information about new or changed functionality

• List of bugs included

– Functional Impact Assessment document • Functional changes

• Functional impacts

• Testing recommendations

14

Plan




15

Evaluate

• Task on hand

Review

• Documentation

Scope

• Key business flows

• Impacted Flows

• Usage patterns

Plan




Automate

Execute

Plan



Recommendations & Tools Basics …

17

Execute

Entry / Exit Criteria

Follow A Testing Methodology Based on Task

Theme Based Exploratory Role based End to end flow

based

Mark Test Flows With A Priority

Basic Sanity check Detailed


Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 18

Shorter Release Cycle

Automation coverage

Rapid Addition of Features

Faster validation

Security

Testing data security and data leaks

Integration

Test integrations for Cloud & On Premise

Multi Tenancy

Simulated customers and roles Testing

Recommendations & Tools Cloud Testing …

Execute

Accelerate testing with automation tools

Test in short sprints

Authentication, Authorization, Input Validation, PWD, etc ..

End to end flows between the cloud apps and integrating apps including on premise ERP

Ensure isolation of individual customer applications and data

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 19

Complex Architecture

Component specific & integration

testing

Complex Data

Controlled statistical data

with pre-defined patterns

Security

Data security, leaks, and

authentication

Validation

Insights & predictions VS. Pre-validated

data

Performance and Scalability

Performance and stress

testing

Recommendations & Tools IoT Testing …

Execute

Test different services separately and run end to end application flows

Feed data into the IOT apps to see if system can decode the time series data

Authentication, Authorization, Input Validation, PWD, etc ..

Feed same data to IoT apps & data mining tool to compare results

Test with volume data & multiple users


Recommendations & Tools Good Practice

• Be creative / Innovative

– Review results

– Learn from results • Hot spots

• Safe spots

– Plan next cycle with a new flavor • Shift focus to hot spots

• Certify safe spots

– Improve scoping

20

Execute




Automate

Execute

Plan




• Manual Vs. Automation

• Reduce testing time & effort

• Repeatable!

22

Automate




23

Anyone

Critical + Repeatable Flows

It is 2016

Automate



• Load and Performance Style Testing • Web and Oracle Application Testing Accelerators • Smart Integration with EM Diagnostics

• Test Planning, Requirements and Test Cases • Defects Tracking • Reporting

• Functional and Regression Testing • Web and Oracle Application Testing Accelerators • Oracle Flow Builder and Starter Pack

Oracle Application Testing Suite Overview

Oracle Functional Testing Suite for Oracle Applications

Oracle Load Testing Suite for Oracle Applications

Oracle Test Manager

25

Automate



Oracle Functional Testing Suite for Oracle Applications Oracle Flow Builder and EBS Pre-built Test Content

• Use Flow Builder to Generate Test Scripts

• Start with EBS Pre-Built Test Content

• Create and Maintain Your Test Scripts

– Add Your Flows & Data

– Extend with More Components

– Support Customizations

– Non technical resources

Record Manually

Generate

Test Automation

Scripts

OpenScript

Record & Playback + Advanced Scripting

Flow Builder

Framework to combine flows and data to generate

test scripts

EBS Pre-Built Test Content

Your Data

Your

Flows

Ora

cle

Fu

nct

ion

al T

est

ing

Suit

e

for

Ora

cle

Ap

plic

atio

ns

EBS Quality Assurance Team Reduced Automation Time by 60%

26

Automate



Oracle Flow Builder: EBS Pre-Built Test Content Starter Pack

• Commonly used base functionality

• For creating high level certification flows

• Broad coverage to jump start the testing

For Both 12.1.3 and 12.2

2000+ Components and 200+ Flows across EBS

Procurement Financials

Manufacturing Logistics Service Master Data Management

Order Management

27

Asset Lifecycle

Management

Projects HCM

3600+ components and 230+ flows across EBS

Logistics Manufacturing Financials

Projects Procurement

Product Information Management Service

Human Capital Management

Order Management

Asset Lifecycle

Management

Automate


Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack

• Niche products which stand alone and deep functionality

• For creating detailed and complex test flows

• Significant number of components above Starter Pack (95% coverage)

For Both 12.1.3 and 12.2

28

10 product families / products across EBS

Incentive Compensation

Discrete Manufacturing

Channel Revenue

Management

Project Management Procurement

Process Manufacturing

Property Manager Service

Lease /Finance Management

Asset Lifecycle

Management

Automate


Oracle Flow Builder: EBS Web Services & APIs Support

• What are Web Services & APIs?

– Stored procedures that enable you to insert and update data into Oracle Applications

• Automate Web Services & APIs using OFB

– No need for an in-depth knowledge of PL/SQL

• Advantage – Automated mass upload into the application

– Scripts created are more stable and robust than UI

29

Automate



Automate

Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack includes content for API/Web Services

• Drastic reduction in test execution time

• Hybrid flows, where you can execute a flow with API and UI scenarios

• Verification of data in UI, Database, or output data after interface execution

Enhanced Content for 12.1.3

30

875 components and 48 flows across EBS

Logistics Manufacturing Financials

Projects Procurement

Product Information Management Service

Trade Management

Order Management

Asset Lifecycle

Management

API Content for EBS 12.1.3


Oracle Flow Builder: EBS Web Services & APIs Support

• UI VS. API

– Value added through execution • In Oracle Project Management:

– Creating a Project takes 1 - 1.5 hrs using UI mode VS. 26 seconds using OFB API

– Create Task Assignments takes 15 min using UI mode VS. 8 seconds using OFB API

31

Automate



Demo Oracle Flow Builder - APIs

Oracle Confidential – Internal/Restricted/Highly Restricted 32

33 Qualcomm Proprietary and Confidential

Testimonial from Qualcomm

• Created ~115 custom components in Oracle Flow Builder which saved huge effort during design phase.

• Created ~160 automated scripts which cover ~1470 business process scenarios in past ~2years

• Metrics against manual testing that is qualified for automation: ◦ Time Savings with Automation ~50%

◦ Automation Script Coverage ~50%

• Automated scripts used in below activities which reduced a significant manual effort during major projects: ◦ Regression cycle

◦ Quick Build test

◦ Month-end activity close

◦ Pre-production checkout

◦ User Acceptance test

◦ Go-live checkout

◦ Emergency test in PRD Support

34 Qualcomm Proprietary and Confidential

Testimonial from Qualcomm

• Custom Solutions: ◦ We use QC for test management. Created a plug-in in order to interface OATS scripts directly to update results in QC.

◦ For accelerated test execution created customized framework in order to execute same OATS script in parallel for

different business scenarios


Program Agenda

EBS Evolution

Testing Readiness


Customer Story

1

2

3

4


Testing Oracle E-Business Suite

Best Practices & Lessons Learned

Sarvesh Tiwari

Lead Solutions Architect

[email protected] @ideating

http://twitter.com/genpact_ltd

37 © 2015 Copyright Genpact. All Rights Reserved.

68,000 +

employees 800+ Clients

(One Third of Fortune

500)

30 languages

17 delivery

countries

70 delivery

centers

50 + Oracle Go Lives /

year

BPM Analytics Digital Technolog

y Reengineer Consulting

Outcome driven, Process-

led ERP

Analytic Solutions & Insights to support

executive decisions

Using Lean Six Sigma for process

improvements & Transformations Robotic and Digital technologies While running some of the most

complex business processes for

our clients

Deep domain consulting for business

blueprints

AT-A-GLANCE Founded by GE in 1997

NYSE (“G”)

$2.5 Billion (FY2015)

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjM6JiP8ZbPAhVBSiYKHRMVBbYQjRwIBw&url=http://logos.wikia.com/wiki/File:500px-General_Electric_logo_svg.png&bvm=bv.133178914,d.amc&psig=AFQjCNEMOsOenD-p9uNagViEuwX4mzW6aA&ust=1474218241217969

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=0ahUKEwj73OT4kJ_PAhVMQyYKHZZBDSIQjRwIBw&url=http://www.dbi-services.com/our-it-expertise/partnerships/oracle-platinum-partner/&psig=AFQjCNEi0Sq2z0A0oEIYP84SZRUELcDLxA&ust=1474501645856081


FUTURE IS HERE, NOW

Computing Everywhere … EBS on wearables?

Internet of Things … sensor to boardroom

Advanced Pervasive Invisible Analytics … In-memory

Context Rich Applications UI … tried Endeca?

Cloud-Client Architecture … mobile native

Infinitely Elastic Infra … cloud compute…


IT Practices

Business

Technology

17% CIOs say they are fast enough for the pace of their business

“IT Speed: The Crisis and the Savior of the Enterprise” Forrester Research

CRISIS


ORGANIZATIONS ARE LEARNING FROM SILICON VALLEY

LEAN START UP | AGILE DEV + OPS

TEST DRIVEN DEVELOPMENT | CONTINUOUS TESTING

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiboZSf3ZbPAhXFQiYKHdWADjkQjRwIBw&url=http://reqtest.com/testing-blog/tdd-and-atdd-an-overview-of-the-two-popular-methods-2/&bvm=bv.133178914,d.amc&psig=AFQjCNGVjGlqtnpCtdVIbWLaGEoYFJszmQ&ust=1474212827914557

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiso9S33pbPAhUGbSYKHVkpBwcQjRwIBw&url=http://www.midfinsystems.com/devops/&bvm=bv.133178914,d.amc&psig=AFQjCNEw-FNsnQbrOBzoyq1ALb5mZ81WSw&ust=1474213188384910


HOWEVER, MOST FACE SPEED BUMPS IN TESTING

45% of effort is spent in creation of prerequisites and

documenting, rather than testing the outcomes

53% ERP Programs execute EBS testing as “part

time” function of the business end users

62% report having no automation in EBS Functional

Testing

43% CRITICAL PATH OF A TYPICAL EBS PROJECT IS IN TESTING

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwid_fmb6ZbPAhWHWCYKHZuxDMoQjRwIBw&url=http://www.abetterinterview.com/part-time-jobs-that-will-boost-your-resume/&bvm=bv.133178914,d.amc&psig=AFQjCNGP5ZeZEcRG0vZ-_eTWm1oZVFIlQQ&ust=1474216130966937

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwix-tS16pbPAhUCTCYKHe54AQwQjRwIBw&url=http://auto.howstuffworks.com/auto-racing/nascar/nascar-basics/nascar-tire4.htm&bvm=bv.133178914,d.amc&psig=AFQjCNEnKuU525zCetd410V_pQLgJj4RSA&ust=1474216397225827


LEARNING FROM A JOURNEY

US headquartered global leader in data center solutions, cloud

management, performance management, services management,

workload automation and virtualization lifecycle management products

& services

• Complex global EBS footprint (Fin, SCM, CRM)

• 58 SOBs, 40 countries, 13 localizations

• 20+ modules, 30+ integrated systems

• Heavily customized (3,500+ CEMLI)

• 5 custom-built bespoke bolt-on modules

• Products shifting to SaaS

• Failed attempts to upgrade to R12.2

• Heavily manual-dependent testing effort

• Painful release cycle time: want to establish monthly

releases

CONTEXT CHALLENGE

CLIENT


5 TOP CHALLENGES THEY FACED IN AUTOMATING EBS TESTING

• Domain Knowledge and Testing knowledge mutually exclusive impacting speed, quality

and effectiveness of testing

• Too much cycle time it takes to develop automated test scripts

• Always Obsolete: Scripting based test cases /data become easily obsolete therefore

require huge ongoing effort in keeping scripts up to date.

• Difficult to discover / reuse the test data, scripts, scenarios from release to release, project

to project

• Difficult to scope and tailor (Test some / Test whole) for individual releases


JOURNEY

Oct 2014

Dec 2014

Mar 2015

June 2015

Sep 2015

Nov 2015

Jan 2016

R12 Upgrade Kick

Off

OATS Pilot

1st OATS MVP

Ready – Used in

Unit Test

Progressive

Additions in

OATS

R122.2 Gone

Live

Monthly

Regression

Pack

Test Iterations Ongoing Optimization


RESULTS

•50% faster test iterations

•40% Less business effort in test cycles: Focus on value-added

test

•60+% reduction in regression cycle times

•Production releases: 8 /year (from earlier 4 /year)

•Adoption of Agile and Test-driven development methods

•100% Sanity, 80% Regression, 60% “projects” EBS testing

automated


LESSONS LEARNED: BALANCE OF MANUAL & AUTOMATED

Low Testing

Effort

Repeatable / Rule Based

High Testing

Effort

High Intuitive

1 2

• Automate the pre-requisites, support SMEs to focus

on “real” validation

• “Auto-gathering” of test evidence / progress: End to

End scenarios

• Role Based Testing

3

Limited value: diminishing returns of automation

Low Hanging Fruits

Automate to sustainable extent

X


BUSINESS-DRIVEN DOMAIN KNOWLEDGE LED

Script-free;

Drag & Drop;

Keywords;

Tags


PRE-BUILT TEST CONTENT: AGILE JUMP START

Business Scenarios

• End to end test scenarios covering

detailed critical business

processes, use cases and industry

nuances

• Context specific functional

scenarios

• COA change

• R12.2 upgrade


DESIGN FOR MAINTAINABILITY: MINIMIZE OBSOLESCENCE

•EBS Configuration Half Life:

•Data / Set Ups: 3 - 6 months

•Functionality/Logic: 6 - 12 months

•UI / major patch: 9 – 18 months

•Continuously updated library of test content

•Not “Screen Placement” but Object-Action driven

• “Keywords > Component > Flow” | upward bubbling of changes

•Changes in scripts for EBS customizations


HOT SPOTS VS SAFE SPOTS

•Test Clustering: L1 … L5 end to end scripts

• Iterative Learning: Progressive adaptability – Hot Spots vs Safe Spots

• Impact Traceability to Test Scenarios

•Test Driven Enhancement Management

•Continuous Optimization: “Regression Packs”, “Sanity Packs”

•Test Data

•Role based Focal Data Points (PO data vs Accounting data)

•Data Randomization, Variability

•Purpose-driven Data Generator scripts

Thank You

Sarvesh Tiwari

[email protected] | @ideating

mailto:[email protected]