unleash service virtualization: reduce testing delays

T3 Test Automation

5/2/2013 9:45:00 AM

Unleash Service Virtualization: Reduce

Testing Delays

Presented by:

Allan Wagner

IBM Software - Rational

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Allan Wagner

Allan Wagner is a technical marketing manager and evangelist with IBM Rational, driving thought leadership, strategic initiatives, and tangible solutions around the adoption of Rational ALM for IT and manufactured products. Focusing specifically on quality, requirements, and test management during his ten years of practical field experience, Al has assisted, mentored, and enabled both internal IBM and external customer teams to address their IT application infrastructure, development, implementation, and operations challenges. Al is a frequent conference speaker on topics of software quality products, principles, and techniques and has authored numerous papers.

1

© 2006 IBM Corporation

Unleash Service

Virtualization Reduce Testing Delays SC051

Al Wagner [email protected]

Technical Marketing Manager

2

IBM Rational Software

“Unleash Service Virtualization” agenda

What?

Why?

How?

When and Where?

2

3


Service Virtualization is…

a method to emulate the behavior of specific

components in heterogeneous component-based

applications such as service-oriented architectures.

used to provide software development and QA/testing

teams access to dependent system components that are

needed to exercise an application under test (AUT), but

are unavailable or difficult-to-access for development

and testing purposes.

a method to emulate the behavior and performance of

specific components in heterogeneous component-based

applications such as service-oriented architectures.

4


Service Virtualization enables you to

create “virtual components” which…

can simulate the behavior of an entire application or

system during testing.

can run on commodity hardware, private cloud,

public cloud.

allow each developer, tester to have their own test

environment.

allow developers, testers to continue using their

testing tools (Manual, Web performance, UI test automation).

http://en.wikipedia.org/wiki/Service_virtualization

3

5



What?

Why?

How?

When and Where?

6


Complexity Multiple layers of

technology

Multiple vendor

platforms

Complex

transactions

Complex

dependencies

Multiple

stakeholders

4

7


Testing this complexity is hard! Application complexity is exploding:

Mainframe (one interface)

Client Server (a few interfaces)

Web/SOA/Cloud apps (100’s of interfaces)

Development and Test teams are getting

larger and more geographically disperse.

Costs are escalating and quality is

suffering.

Waterfall models and serial projects are

out, agile and continuous testing are in.

A new approach is required to keep up!

8


And yet, we test the same way

8

Do you know

what’s under the

surface?

5

9


The black box approach…

10


10

What if you don’t have all the pieces?

6

11


Blockers of early end to end testing…

“The environment

refresh takes

3 weeks!!”

“It is too expensive to

setup a live message

feed for test!”

“Can we begin testing

before everything

is available?”

“The dev team

hasn’t started

work on it yet!”

“We’re still waiting for

them to deploy?”

12


Does your organization look like this?

Integration Testing lags a

step behind the code

Tester

Setup integration test environment

Developer

Write & deliver code

Nightly build(s) Compile, unit test, publish

Developer

Developer

Developer

Developer

Developer

N days to install & configure

I just spent 2 weeks setting this up &

the build is no good??

Iteration 1 Design, Code, Test Weeks 2 - 6

Integration test

iteration #1

Iteration

planning

Week 1

Iteration 2 Design, Code, Test Weeks 8 -12

Integration test

iteration #2

SVT

prep

Iteration

planning

Week 7

When agility stops at integration testing, the entire

organization hits the wall

N nightly builds are piling up

7

13


Service Virtualization brings opportunities

13

Challenges Opportunities

Drive testing earlier, discover defects early

when they are cheaper to correct

Automate testing that matters freeing up

resources for other value add activities

Improve resource governance and control

Long cycle times

Availability of resources

Escalating labor costs

High cost of defect fixes

Poor asset utilization

Inadequate testing and test

environments

Defect escapes

Test data consistency

Time To Market

Cost

Quality

Reduce infrastructure required to develop

and test applications

Increase resource flexibility and availability

Improve asset utilization

Provide rapid delivery of services

Allow testing when only portions of the

final solution are available

Increase levels of testing

14



What?

Why?

How?

When and Where?

8

15


Ad-hoc stubs or “mocks”? What is an ad-hoc stub or “mock”?

– A piece of code used to stand in for some other

programming functionality.

– A stub may simulate the behavior of existing code or be a

temporary substitute for yet-to-be-developed code.

This approach is:

• Labor intensive to write and maintain mocks.

• Requires deep knowledge of both technology and the

business logic to be emulated.

• Requires changes to application configuration when

switching to live services for testing.

• Hard to share across the entire team.

16


Six steps to Service Virtualization success

Model Configure Discover Deploy Execute Virtualize

Execute

specified test

cases

Listen and

capture

transactions

Build

negative test

case

Refine ROI

Model

behavior of

transactions

Model

negative test

cases

Model

“perfect

storm” test

cases

Apply

production

metrics to

virtual models

Deploy

models to

virtual

environment

Configure

SUT

connections

Execute

specified test

cases

Capture test

results

Capture and

measure

results

Recommend

AV next steps

Install and

configure AV

software

Configure

transactions /

Protocols

Configure

Services

Build

additional AV

test cases

Identify

testing pain-

points

Understand

IT

technologies

Review Use

cases

Initiate ROI

Analysis

Understand

testing

methodology

9

17


Discovering what to virtualize

Technology Component Impact of

Unavailable

Complexity

Score

Protocol and

message format

Component

Name

Dollar cost of

impact

Scaling Factor

(1-10, low – high)

Examples:

Web Service -

Soap over http

Credit Check

Service

$10,000 2

Web Service -

Soap over http

Payment

Service

$60,000 2

SAP Journal

Entry

$100,000 6

18


Virtualizing the “real” behavior

Category Detail

Simple Hard-coded response returned for given input

Non-deterministic One-of-n hard-coded responses

Data driven Input and/or output data specified in external

data source (Excel, file, database)

Data model, stateful

Input and/or output data kept in data model with

complex relationships. Supports CRUD and

other stateful behaviour

Behavioural

Extends model-driven to provide pre-packaged

functionality, e.g. shopping basket, real-time

data feed, trading exchange, order matching

1

0

19


Listen and capture the behavior Physical Test Environment

Volume Constraints

Volume Constraints

2hr. access window

Maintenance Constraints

Database / JDBC

Server/ Web Services

Mainframe / MQ/CICS

Smart Device / SD-IO

MQ/CICS

Web Services

Smart Devices

JDBC

Virtual Environment

Capture & Model

System Under Test

Listen / Capture

Store

Test Case

Repository

20


Build a system model from recorded events

Queue.A.Reply

Queue.A

Application

Servers

Clients Queue.B.Reply

Queue.B

Queue.C.Reply

Queue.C

1

1

21


Tu

rn s

pe

cific

atio

ns in

to v

irtua

l co

mp

on

en

ts

Build virtual components from design specs

HelloService

22


Synchronization helps respond to change Synchronization provides an interface for importing

project assets, and mirroring updates to those assets.

1

2

23


Service Virtualization allows you to… Eliminate critical development and testing constraints by

virtualizing IT resources

Deliver realistic simulated development and test

environments at a fraction of the cost

Isolate defects and drill into their root cause

Create automated regression test suites to reduce test cycle

time and improve test coverage and quality

Leverage Cloud services with Service Virtualization to

minimize infrastructure set-up time and provide scalable, on-

demand infrastructure

Drive benefits across the entire application development and

test lifecycle

24



What?

Why?

How?

When and Where?

1

3

25


Service Virtualization across the SDLC

Define

constrained

systems and

services for

virtualization

Integration Operability User Acceptance System Integration System Unit Integration Operability User Acceptance System Integration System Unit

Performance

Developer uses

virtual services to

unit test code on

their workstation

– no elaborate

environment

required

Development and Test teams

uses virtual services for

early stage advanced testing

The full system or

components of the system

can be performance tested

early in the cycle. Virtual

services can be used when

components are not

available

Later in the cycle, final pre-

production versions of

services are swapped in as

they become available. Final

testing is done against real

services

26


The old world

OT

Performance

UAT

Sys Tests

Unit Tests

Requirements Quality Gate

Big Bang

• Multiple defects introduced at once

• More unknowns – greater risk

• More expensive

1

4

27


The new world

Performance

Requirements

Unit Tests

UAT

OT

Integration Tests Sys Tests Sys Tests

• Accelerates testing

• Reduces costs

• Lowers risk Big Bang

Shift testing to the left!

28


Incremental integration testing example

C1 C2 C3 ERP WSDL 3rd party Database Pass/Fail

Real V V V Test my

own piece

• Test C1 with three virtualized services.

• Quick to setup and low-cost.

• Author integration tests early

1

5

29


Time

Real V V V R

Real V V V V

Real V V V V V V

Real V V V V R R

Test my

own piece

Integrate

w/another

This won’t

slow me down!

First Complete

System Test

All components

Integrated!

C1 C2 C3 ERP WSDL 3rd party Database Pass/Fail

Real V V V


30


Pass/Fail Time

Real V V V

Real V V V R

Real V V V V

Real V V V V V V

Real V V V V R R

Real R R V V R R

Real R R R R R R

Test my

own piece

Integrate

w/another

This won’t

slow me down!

First Complete

System Test

All components –

Integrated!

The expensive

backend

All Systems UP!

• Gradually integrate with expensive

backend systems.

• Minimize use of backend systems for

testing purposes

• Process offers easy and cost effective

defect isolation

• Integration tests are written and

executed earlier

C1 C2 C3 ERP WSDL 3rd party Database


1

6

31


Changing the performance test model

Requirements Design Coding SIT UAT Production

MONITORING

PREDICTIVE

ASSESSMENT

ASSESSMENT

Low Intensity

Performance Testing

(LIP)

High Intensity

Performance Testing

(HIP)

Incremental integration strategy defines HIP start point;

focus of HIP is informed by LIP results

32


Streamline your process – test continuously Avoid time spent installing and configuring software just to

discover basic build issues

Smoke tests can be integrated as part of the continuous integration

process through deployment automation and service virtualization

Regression test results are made available to the entire team to

shorten resolution

Assets Product

Increment

Virtual Machines

App Server

Order Mgr

Virtual Lab

Manager

Cloud Environment

CRM

VM Catalog

Build •Smoke Tests

•Sandboxes

•Self-provisioning

1

7

33


http://ibm.co/servicevirtualization

34


© Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.