smart automation & driving intelligence into qa...designer test data ca test data manager...

1 us.sogeti.comCONFIDENTIAL AND PROPRIETARY INFORMATION. © 2018 SOGETI USA LLC

Executive Summit 2018

Smart Automation & Driving Intelligence Into QA

Sathish NatarajanGroup Vice President – Digital Assurance & Testing, Sogeti

Greg MeuseGlobal Solution Advisor, CA Technologies


In this session you will learn about…

ü Executive Insights – World Quality Report• Objective analysis of trends shaping Quality Assurance & Testing function

ü SMART Automation and Driving intelligence into QA• Continuous Testing and Intelligent Automation leveraging CA Technologies

3 us.sogeti.com

4 us.sogeti.com

Research – Focus Areas

321

Digital Transformation Automation

Agile and DevOps

5 6

Test Environment and Test Data Management

QA and Testing Budgets

4

Industrialization

5 us.sogeti.com

Interviews by Region

6 us.sogeti.com

Interviews by Sector

7 us.sogeti.com

Interviews by Job Title

8 us.sogeti.com

Key Findings and Highlights

1. Digital Transformation: 41% respondents see ‘Increasing the Quality of Product or Software’ as most important QA & Testing objective

2. Agile & DevOps: The challenges with testing in agile development are increasing

3. Test Automation: on the way to smart, intelligent, and cognitive QA. Automation is currently un-exploited in QA & Testing (16%)

4. Industrialization: QA organization evolving to meet bi-modal needs. Shift from TCOEs to TECs

5. Test Environment & Data: Test environments and test data continue to be the Achilles heel for QA and testing

6. QA Budgets: Test budgets fall but are expected to rise again; from 35% (2015) to 26% (2017), will rise to 32% (2020)

9 us.sogeti.com

In Summary

QA Trends across industries

04030201

Contain Cost of Quality

AI/ML in testing Responding to Bi-Modal Needs

Focus on Quality

Engineering

Key Focus Areas

Intelligent Test Automation and Smart

Analytics

Application of Analytics/ML/AI

Worforce Transformation

QA to QE transformation is rapidly taking place in fast-changing digital world


Driving Intelligence into QA

Continuous Testing and SMART Automation


Every Business is in the Software Business

Delete apps that load slowly2

75%

Expect app to load in under 2 secs.1

49%BORN DIGITAL

TRANSFORMING DIGITIAL

1.Akamai, “Performance Matters: Key Consumer Insights”2.Dynatrace Holiday Shopping Report


Those Who Don’t Move Quickly Will Be Disrupted

https://en.wikipedia.org/wiki/Category:Companies_that_filed_for_Chapter_11_bankruptcy_in_2017

Major Brands That Have Filed Bankruptcy in 2017


Eliminate the barriers between ideas and outcomes

Software businesses must


However, Traditional Development Practices Slow Ideas from Becoming Outcomes

62%of time spent on un-planned/routine work 2

70%of all testingis still manual 3

57%Dissatisfactionwith deploy times 4

64%of defect costs originate in planning 1

80%of resolution time spent ID’ingissues 4

Idea Value

Plan Develop Test Release Operate

1. Crosstalk, the Journal of Defense Software Engineering2. 2016 State of DevOps Report3. Bloor Research4. Source: Forrester Total Economic Impact,


A Modern Software Factory Requires Continuous Delivery

Continuous Delivery

Idea Value

62%of time spent on un-planned/routine work 2

70%of all testingis still manual 3

57%Dissatisfactionwith deploy times 4

64%of defect costs originate in planning 1

80%of resolution time spent ID’ingissues 4

Plan Develop Test Release Operate

30 – 50%Improved delivery

speed over waterfallDecrease in Test Time

20xIncrease in

Deployment Speed

+80%Reduction in

Outages

90%


Continuous Delivery

Idea Value

Accelerate and streamline development, testing and

release by automating the SDLC

Continuous Delivery Continuous Development

Continuous Release

Continuous Delivery Requires Continuous Everything

Plan

Develop

TestRelease

Oper

ate

Continuous Testing


The Way We Look at Testing Needs to Change

Despite investment in automation, 70% of testing today is still manual—a new approach is needed.

But testing needs to be embedded in every process.

Plan Develop Test Release

Traditionally Testing has been an event in the SDLC

Sprint Sprint

Test

Plan Develop

Release

Test

Plan Develop

Release

Test

Plan Develop

ReleaseSprint


Continuous Testing is the Answer

The embedding of testing as a fundamental and ongoing aspect of every activity though the software delivery cycle.

PEOPLE

Empower developers with testing capabilities.

Leverage open integration and cloud.

cPROCESS

Automate everything.

TECHNOLOGY


Developers Need to Be Empowered to Test

Testers evolve to automation engineers

Shift testing left to developers

Democratize testing tools


The Core Elements of Continuous Testing Technology

Open Integration and Cloud-based

Virtualized Environments

& Test Data

End-to-end Automation

Orchestration and Analytics

RequirementsModeling


RELEASEPLAN

Agile Development Platform• Managing users• Providing Agile at Scale capability• Housing artifacts (user stories, test cases

with results)

LOAD GENERATOR

SAAS PORTAL

~

TEST DATA

TEST CASES

VIRTUAL SERVICES

APP

Model-based Testing• Auto-generate test cases• Optimization of test cases

Service Virtualization• Virtual services emulate

unavailable systems• Reduce infra and 3rd party

costs

• Performance testing from dev through production Performance Testing

DEV,TEST

Test Automation Framework• Automate Unit, Functional, Load, & Regression

Test Data Management• Create, store, track, share and re-use test data• Virtual or real data stores

Continuous TestingHow do you build better apps, faster?

Application Security Testing • Coverage of entire SDLC from earliest stage

of development through testing and production

Open Integration

Application Release Management• Intelligently and Automatically Moving Artifacts and

assets thru the SDLC

CA Agile Central

CA Service Virtualization

CA Agile Requirements Designer

CA Veracode

CA Test Data Manager

CA Application Test

CA Blazemeter

CA Continuous Delivery Automation


DEV,TEST

CA Application Test Test automation runs Automic and UIPath tests

Intelligent Automation – (R)evolution How do you Automate your Automation Factory?

Service Requests

Business Processes

TEST RESULTS

UserStories

Sprints

Releases

CA Agile Central

MODELCA Agile Requirements Designer

TEST DATA

CA Test Data ManagerGenerate Synthetic data for tests

TEST CASES

VIRTUAL SERVICES

Create and use Virtual 3rd

party services

CA Service VirtualizationUIPath Testing

Framework

CA Continuous Delivery DirectorPlan, orchestrate, and gain visibility into the application delivery lifecycle from requirements to production

PLAN

CA Automic Release Automation

Deploy, track and fully automate releases of Bots

RELEASE OPERATE

CA Digital Experience InsightsClient

Using your Modern Software Factory to automate your ITPA and RPA Factory

CA Automic Testing

Framework

Model requirements to automate test creation and optimization


What does agile data do for you?

Steer the business with ease by

leveraging data in one place with a

standardized format.

Centralized Steering

Align execs, depts. and teams on top

priorities with transparency and visibility to focus

on better business outcomes.

Catch cadence, velocity, budget and

dependency issues in real-time. Make

corrections before they impact time to market.

Better Business Alignment

Actionable Delivery

Predictive Plans

Leverage real-time, historical, and trend data in one place for insight into delivery

times–enabling teams to better gauge future

success.

Back


Use Model Based Testing to Drive Agile Testing

In Four Use Cases that Matter

Automatically generate for all functionalities being delivered

optimized tests in every sprint.

ACHIEVE IN-SPRINT EVERYTHING WITH

AGILE TESTING

Back


Test Data Management – A Comprehensive Approach

PRODUCTION DATA EXISTING DATA TEST DATA ON DEMAND

A complete solution:• Profile, subset & mask existing data

• Find and Reserve your data

• Generate test data synthetically in the format

needed

• Deliver data virtually with vTDM

The result:• Test data when your testers need it

• No more waiting for test data

• Better data in the formats needed

• Higher quality, compliant data

Locate Data across

many environments

Visualize &

analyze data

quality

Deliver Data in

quantity that

tester needs

Copy, clone or

generate data to

target

Back

Back


Service VirtualizationHaving Everything you need, when you need them.

Front End Back End

Internet of Things

Mobile

SaaS/Cloud

Partner Ecosystems

External Developers

Secure Environments

Applications

ERP/SAP

Mainframe

3rd Party/APIs

Orchestration Layer

Performance Testing§ Provide component level performance testing§ Test negative performance conditions§ Shift performance testing left

Integration Testing§ Remove constraints with virtual services§ Avoid manual repetitive development of stubs/mocks§ Simulate web services, Middleware, 3rd Party, APIs, or

Mainframe

Better Quality§ Test sooner and more often § Increase test coverage and negative testing capabilities§ Find up to 90% more defects at least one cycle earlier

Faster Time to Market§ Always available resources to continue work§ No waiting for backend or 3rd party resources needed§ Speed Software Development Lifecycle by up to 50%

Back

Back

DevSecOps: Uniting Development and Security

Code Commit Build Test Release Deploy Operate

Operational Application SecuritySecurityAssurance

Secure SoftwareDevelopment

SECURITY DRIVEN

Emphasis on regulatory and compliance requirements

Focus on internal and third-party risks

Prevention Governance Protection

DEVELOPER DRIVEN

Emphasis on immediacy of results, automation and

integrations

OPERATIONS DRIVEN

Emphasis on vulnerability management and protection

Back


Test Automation FrameworkComplete test coverage for Agile Practices … Mobile to Mainframe

KEY BENEFITS• Use for API, Rest API, Mobile, Swift, and SAP testing• Presentation layer test automation for Web & Mobile UIs

• Reporting and collaboration capabilities that allow team

members to execute, monitor and analyze test results

More that GUI Testing! Invoke and verify the behavior of

each layer that makes up an application

Automate Unit, Functional, Load, Regression & Performance Testing

Beneath the UI … lies the complexity to be tackled

Back


What Shift Left & Right Means To You

Test Performance of new code

Test Performance of every deployment (i.e. staging)

Test Performance of every build

Test Performance in production

SHIFT LEFT SHIFT RIGHT

BackBack


WHATModel, abstract and

track applications

Applications/Services

Models

User Stories

Model-driven Releases for All EnvironmentsWork from a common release blueprint

HOW

Standardize the release practice across application stacks.

Reduce release set-up and design time, plus reuse releases.

Flexibly design release models to fit all application types.

Abstract release processes from environments so you aren’t locked into a specific cloud provider.

Manage releases in hybrid environments–both on-premises and cloud.

RESULTImproved quality and speed of application releases: 3 per day from once every 2 days

“…we have been able to reduce manual intervention, increase reliability, gain consistency across our environments, orchestrate parallel deployments and reduce downtime while meeting /exceeding SLA uptime.”

–Charter Communications

Provision & decommission on-demand

QA/TEST

DEV

PRODUCTION

PRE-PROD

WHERE

Back


Appendix

32 us.sogeti.com

Digital Transformation: Out of the following list, please rank the top two objectives that are most important when it comes to Testing and Quality Assurance ?

In digital business, we must manage the dependencies between quality, security and customer satisfaction to assure success

33 us.sogeti.com

Agile & DevOps: Challenges faced in applying testing to agile development

The promise of agility & scale must be balanced with quality

34 us.sogeti.com

Agile & DevOps: Changes required in QA & Testing skill-sets on account of agile and DevOps adoption

35 us.sogeti.com

Industrialization: Approaches to Test Centers of Excellence

How Test Centers of Excellence adapt to the digital era

36 us.sogeti.com

Test Data & Environments: Challenges with Test Environment Management

Test data and environment requires velocity and low cost set up to support agile development & Digital businesses

37 us.sogeti.com

QA Budgets: Proportion of total IT budget allocated to QA & Testing

By 2020, QA will take up 32% of the IT budget

38 us.sogeti.com

QA Budgets: Proportion of Testing & QA budget allocated to hardware, infrastructure, tools, and HR

39 us.sogeti.com

Test Automation

Smart Automation: The key to meeting increasing demands on QA in the digital world

40 us.sogeti.com

Key Challenges in achieving desired level of test automation

41 us.sogeti.com

Emerging Automation Techniques

smart automation & driving intelligence into qa...designer test data ca test data manager...

Documents