www.ranger4.com

DevOpstastic

WebcastMaking Production Like Test

Environments

Helen Beal Nick Blamey

www.ranger4.com

DevOpstastic

2

www.ranger4.com

DevOpstastic

3

Traditional Testing: Build prototype, then try to break

the car after assembly.

www.ranger4.com

DevOpstastic

End to EndTest

Most Issues are found by QA / Load Testers prior to going live.

Defects / Issues Found

. . .

Pre-Service Virtualisation

Production

www.ranger4.com

DevOpstastic

5

Google: “How to build a supercar + McLaren”

Shift Left: Find Defects Earlier: Final End to End Test = Fine TuningSaves Cost, Isolate defects pre-production

www.ranger4.com

DevOpstastic

Most issues are found and fixed within a Sprint.

Defects / Issues Found

. . .End to End Test becomes an Assurance exercise rather than Testing

End to EndTest

Post-Service Virtualisation

Production

www.ranger4.com

DevOpstastic

7

Challenges

Agile: new demands on

testing

Mobile / Consumerism: Higher project

throughput (compressed

SDLC)

Complex ecosystem

of IT Delivery

Multiple stakeholders

Faster identification

of defects

Escalating cost of quality

www.ranger4.com

DevOpstastic

Presentation layer

Integrations, data and business logic

Only one-tenth of an iceberg is above water

Traditional UI testing:

Automated Integration Testing and Service Virtualization:

The majority of risk in modern systems is seldom tested as it is unseen.

www.ranger4.com

DevOpstastic

WebSphere – Example Technology Landscape

FINANCE(CICS)

MANUFACTURING(TIBCO BUISNESS WORKS)

CRM(SOFTWARE AG IS)

TRANSPORT(SAP PI)

XML/HTTP

CICS TG

REST

ABAP

SI Bus

wmBroker

JDBC

RMI

MQ

IBMINTEGRATIONBUS

WEBSPHEREAPPLICATIONSERVER

HTTP

SALES(ORACLE)

HR(WEBSPHERE)

DFDL/TCP

www.ranger4.com

DevOpstastic

Value Statement

Expected Business Case

Reduce software development costs and increase xxxxxxxx capability to deliver projects on time, budget and to the business requirements

Cumulative Savings of £3.84m over 5 years

Average annual benefit

£768k p.a.

The Impact of Testing Earlier Cost reduction by fixing Defects earlier in the software development lifecycle - based on XXXXXXs 15,692 defects from Nov 1st 2013 to Oct 31st 2014. Defects found in the later stages of the lifecycle cause substantially more rework, additional cost and delivery delays compared with finding these defects earlier

Integration Test

Big Bang

AcT Test OATSystem Test

£3,200

£7,680

£97

Reduction in cost of

fixing defects

Earlier testing and Risk mitigation

£97 - Integration Test £250 - System Test£3,200 - AcT Test £7,680 - OATSource: SEI*

£250 *Source: The Software Engineering Institute (SEI) is an independent body http://www.sei.cmu.edu/

www.ranger4.com

DevOpstastic

Sys Tests

Unit Tests

UAT

OAT

Performance

Quality Gate

The Old WorldBig Bang

• Multiple defects introduced at once• More unknowns – greater risk• Expensive!

Requirements

www.ranger4.com

DevOpstastic

Requirements

Unit Tests

Performance

UAT

OAT

The New WorldBig Bang

Integration Tests

Sys Tests

• Accelerated testing• Reduced costs• Lowered risk

www.ranger4.com

DevOpstastic

Concept: Create and Sending of Payloads

Payload

www.ranger4.com

DevOpstastic

Concept: Formating of Payloads

FormatterPayload

Schema?

www.ranger4.com

DevOpstastic

Concept: Use of Schema for Fortmatting

SchemaFormatterPayload SOAP XML

SWIFT COBOL CopybookDTD XSDWSDL HL7EDI IATAText MIMEByte Array Java ObjectsOAG SAP BAP/RFCFIX .Net ObjectsTIBCO ActiveEnterprisewebMethods IB and IS DocumentsCustom…

www.ranger4.com

DevOpstastic

Concept: Use of Test Data

Test DataSchemaFormatterPayload

www.ranger4.com

DevOpstastic

Concept: Use of Transport protocols

TransportTest DataSchemaFormatterPayload JMS webMethods

HTTP/S FTPNDM Oracle/BEAJDBC Sonic MQTCP/UDP PL/SQLFlat Files Shell commands

TIBCO EMS TIBCO iProcessTIBCO RendezvousIBM WebSphere MQCustom Transports…

www.ranger4.com

DevOpstastic

This perspective of RIT shows a logical representation of all available services, servers, DBs, etc. Environments can be

build be mapping the logical representation to different physical

resources.

Model the system

www.ranger4.com

DevOpstastic

Recorded messages. Each color shows a different transport

(MCE, SOA, Saclus DB)

Structure of the selected message.

Format is independent of the message format

(here XML)

Record and re-use live data

www.ranger4.com

DevOpstastic

Testcase created out of the recorded messages. In the case, the test iterates over test date

stored in a CSV file, which is easy to maintain and which could be

reused in other tests/stubs

Hierarchical structure of tests, testdata sets and stubs, grouped by the

different services

Build : Virtual Services, Tests & Performance Tests

www.ranger4.com

DevOpstastic

Execution of a test suite, one of the included tests

failed, one succeeded.

Execute tests, virtual services & performance Tests

www.ranger4.com

DevOpstastic

Report of the test results of a test suite. The links can be used to

jump to a detailed description of a single test. This reports is also

accessible as web page.

Understand the results

www.ranger4.com

DevOpstastic

Library of all deployed stubs. These can be started and stopped with this dashboard.

Deploy for Enterprise re-use of Virtual Services

www.ranger4.com

DevOpstastic

“RBS Markets deploys IBM Rational for virtual testing”Speaking at an IBM Innovate 2013 event in London, Stephen King, Head of Middleware at RBS Markets, said:

We selected IBM Rational Integration Test and piloted a payment compliance application which had to be delivered quickly to prevent the bank facing £5m fines.

Using Rational and the new methodology, the developer team was able to deliver the application in six weeks.

Following the success of the payment application, King said RBS rolled out the methodology and toolset across all the projects the team was working on.

Prior to deploying Rational and the software development methodology, King said regression testing used to take four weeks. It now takes a couple of hours, he said, saving the bank £40,000 in project costs.

Computer Weekly – 7th November 2013 - http://www.computerweekly.com/news/2240208647/RBS-Markets-deploys-IBM-Rational-for-virtual-testing

Key metrics quoted in King’s presentation included:

• 95% increase in project delivery • 99.64% reduction in production incidents • Cost reduction of 63%• Time to market reduced by 40%• 4 week regression tests reduced to 1 hour• Potential fines of 5 million pounds avoided

www.ranger4.com

DevOpstastic

Intangible Benefits

• RTVS will deliver a Common standard for creation of all stubs. A standardized simple and efficient approach to creation of stubs will ensure:

– 100% coverage of all opportunities to create stubs and hence deliver a complete integration testing solution to improve quality delivery

– Stubs are created without need for Development or programming effort – Stubs are re-usable by development and test areas– The process of creation and enhancement of stubs will reduce reliance on scarce

and expensive Subject Matter Experts and Resources e.g. WebSphere MQ + assorted Middleware, SOA Experts

• IBM Rational Test Workbench enables more testing without availability of test environments::

– Some systems are not available the whole time they need to test, therefore, Green Hat VIE can extend test window

– Customer’s 3rd party provided test environment is unavailable / expensive

Centre of Excellence for Stubbing: Creating Testing Assets

www.ranger4.com

DevOpstastic

Intangible BenefitsStart of day testing – • Customer can perform Smoke tests with Rational Integration Tester to ensure that all applications and their components are available and working before starting either the Functional or Performance Testing Process

• This smoke test on completion can then advise Integration / Functional and Performance test teams on what is and isn’t working.

Benefits: • Reduce costs of Managed Test Service: Less Managed Test Service wastage / Consumption based infrastructure charges.

• Data / Batch files are correct prior to load and functional test execution

Smoke Tests to Prove the Environment

www.ranger4.com

DevOpstastic

Continuous Testing Approach

• Test every build• Automatically build, deploy applications to

environments • Leverage virtualization – Use service

virtualization to emulate 3rd party connections

• Automatically provision environments – based on patterns and templates

• Once deployed – Automatically run tests – functional / integration / etc.

www.ranger4.com

DevOpstastic

• System hasn’t been built yet• Components haven’t been delivered yet• Business hasn’t decided which component to buy

yet e.g. AML, Fraud Detection, single sign on, ESB• Two speed IT• Complex Environment with escalating Costs• Data Contention• 3rd party ownership / external stakeholder• Testing window issues

Bottlenecks to Continuous Testing

IBM Continuous Testing is an enabler to solve each of these challenges

www.ranger4.com

DevOpstastic

Be DevOpstastic