Testing anything, any time with containerized service virtualization Leveraging on-demand virtual test environments

for earlier, better and faster testing

Robert Schrijvers & Rix Groenboom

Who are we Joint work:

•  Rix Groenboom (Parasoft) •  Manager EMEA Solution Architects •  R&D with Universities (Groningen, Bamberg) •  Implementing testing solutions for technology, finance, telecom and government

organisations

•  Robert Schrijvers •  Independent Consult •  Java specialist with focus on performance •  Working together with Parasoft since 2012 •  Working for various banks in the Netherlands

Agenda •  Modern times and need for Virtualization

•  Virtualization per phase •  Development •  Test •  Acceptance •  Pre-production

•  Requirements & implementation •  Functionality (Dev) •  Deployment (Ops)

•  Wrap up

Agenda •  Modern times and need for Virtualization

•  Virtualization per phase •  Development •  Test •  Acceptance •  Pre-production

•  Requirements & implementation •  Functionality (Dev) •  Deployment (Ops)

•  Wrap up

Once upon a time …

Business  Logic  

Data  

Client  A   Client  B  q  Large  applica4ons  q  Few  Development  teams  q  Few  Test  teams  q  Few  Opera4ons  team  q  Few  releases  q  Few  Dependencies  q  Life  was  easy  …  but  slow  

Batch  processes  

Modern times

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

Components: Dev -> Tst -> Acc -> Prod

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

D  

T  

A  

P  

DevOps teams •  Many teams

•  Independent

•  Different cycle speed

•  Test often •  Test early •  Test automatically

“Modern  4mes”  

Decoupling by virtualization •  Prevent teams slowing down by other teams

•  Prevent skipping tests in early stages

•  Extend test cases

•  Depends on stage in release cycle •  Various performance profiles •  Various deployment types •  Various transports (protocols) •  Various payloads (messages-types)

Agenda •  Modern times and need for Virtualization

•  Virtualization per phase •  Development •  Test •  Acceptance •  Pre-production

•  Requirements & implementation •  Functionality (Dev) •  Deployment (Ops)

•  Wrap up

Development

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  

Development

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  

Virtualize  N,O  

Development

B  API  

HTTP  D  

T  

A  

P  

Virtualize  N,O  

•  Virtualize on local machine

•  Unit test

•  Functional tests

•  Fully decoupled

•  Fully independent

Test

Data  NoSQL  

A  API  

B  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  C  API  

Test

Data  NoSQL  

A  API  

B  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  C  API  

Virtualize  P,X,Y,Z,  

Third  Party  

Test

Data  NoSQL  

B  API   N  

SVC  

O  SVC  

Y  BA  

TIBCO  

HTTP   TIBCO  

MQ  

PROP  

SOAP  SOAP  

JSON  

PROP  

PROP  

PROP  

D  

T  

A  

P  

Virtualize  P,X,Y,Z,  

Third  Party  

CD  pipeline  

Test  

Test •  Choose “cut-off”

•  Deploy AUT

•  Deploy Neigbours

•  Virtualize •  On Server •  Configured environment •  Supports custom transport •  Supports custom payload

•  Functional tests

•  Environment choice •  Test chain

Data  NoSQL  

B  API   N  

SVC  

O  SVC  

Y  BA  

TIBCO  

HTTP   TIBCO  

MQ  

PROP  

SOAP  SOAP  

JSON  

PROP  

PROP  

PROP  

D  T  

A  P  

Virtualize  P,X,Y,Z,  

Third  Party  

Test  

Acceptance •  Focus on non-functional requirements

•  Component central •  DevOps team responsibility •  Capacity •  Stability •  Performance •  Resilience

Acceptance

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  

Acceptance

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

D  

T  

A  

P  

Virtualize  N,O  

Test  data  

Acceptance

B  API  

TIBCO  

HTTP  SOAP  

JSON  

D  

T  

A  

P  

Virtualize  N,O  

CD  pipeline  

Test  Test  data  

•  Reconfigure •  Slow response •  No response

•  Virtualize •  Reconfigurable •  Performant •  Stable •  Representative data

set

Pre production •  Does not fit in pure DevOps

•  IMHO mandatory

•  AKA “staging”

•  Production like as possible •  Sizing •  Configuration •  Data

•  “Tomorrows production”

Pre-production

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Z  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

Third  Party  SVC  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

P’  

P  

P  

P  

P  

P  

P  

P   P  

P:  Produc-on  version  

P’:  New  version  

P  

Pre-production

Data  NoSQL  

A  API  

B  API  

C  API  

M  SVC  

N  SVC   P  

SVC  

O  SVC  

X  BA  

Y  BA  

Data  RDBMS  

Data  Prop  

HTTP  

TIBCO  

HTTP   TIBCO  

HTTP  

MQ  

MQ  

PROP  

SOAP  

SOAP  

JSON  

JSON  

SOAP  

PROP  

PROP  

PROP  

P’  

P  

P  

P  

P  

P  

P   P  

P:  Produc-on  version  

P’:  New  version  

Virtualize  Z,  

Third  Party  

Test  

Test  

Test  

P  

Agenda •  Modern times and need for Virtualization

•  Virtualization per phase •  Development •  Test •  Acceptance •  Pre-production

•  Requirements & implementation •  Functionality (Dev) •  Deployment (Ops)

•  Wrap up

Requirements gathering (1) Based on these user-stories:

•  Local and Server deployment

•  Multiple Transport protocols

•  Multiple Message payloads

•  Flexible Data-set configuration

•  Run-time deployment

•  Run-time data provisioning

Let’s make a “Separation of concerns” between DEV & OPS

Requirements gathering (2) DEV (building Virtualizations): •  Message responders: XML, JSON, EDI, SWIFT, Text… •  Logic: Synchronous, Async, Stateful, “batch” •  Data: File, CSV, SQL, NoSQL OPS (running Virtualizations): •  Protocol listeners: HTTP, JMS, MQ TCP/IP, File, … •  Deployment Descriptions: Endpoints, Queues, JNDI •  Performance profiles: Delay, Time Out, Errors

•  Some part of the Logic (functionality) •  Some part of the Deployment (transport)

Deployment strategies Two main deployment strategies:

•  Central: Client / Server model •  Repository containing Implementations (PVA-files) •  Environment Manager to control provisioning •  Optional Load Balancing with multiple servers

•  De-central: Container or Cloud model •  API driven launch of local servers •  GIT based deployment of Implementation (PVA files)

Towards completely scripted construction

Central Deployment (client / server)

Virtualize SME

Environment Manager

Stage/Test Server

Production Server(s) Failover Server(s)

Desktop

Application Under Test

De-central Deployment (container based)

Paraso&  Environment  Manager

Towards scripted deployment •  Docker & Java & Tomcat

•  Set protocol jars & Deploy Virtualize (war)

•  Static implementation: •  Deploy PVAs (model) from GIT •  Provision test-data

•  Dynamic implementation: •  Build PVA files on the fly (using API) •  Example:

•  Given “customer with 2 credit cards”

•  When “deleting one card”

•  Then “one card should be left”

Towards scripted deployment

B  API  

TIBCO  

HTTP  SOAP  

JSON  

D  

T  

A  

P  

Virtualize  N,O  

(deployment  descrip4on)  

CD  pipeline  

Test  

Logic&Data  N,O  

•  Pipeline: •  Deploy SUT (eg B API) •  Deploy Virtualize (docker) •  Initialize N,O placeholders

•  Test-execution: •  Populate N, O (“Given”) •  Execute (“when/then”) •  (and repeat)

Deployment summary •  Central server and central deployment:

•  “Waterfall”: Central “test data team”

•  Central server and de-central deployment: •  “Agile”: teams have control over Virtual Assets

•  De-central Server and central deployment •  For 3rd party interfaces / complex backends

•  De-central server and de-central deployment •  “DevOps”: Deploy Virtual Assets as part of pipeline

1.  “library of PVA’s” and provisioning

2.  “on the fly creation” (given-when-then)

Agenda •  Modern times and need for Virtualization

•  Virtualization per phase •  Development •  Test •  Acceptance •  Pre-production

•  Requirements & implementation •  Functionality (Dev) •  Deployment (Ops)

•  Wrap up

Wrap up •  Virtualization decouples your components

•  Decoupling oils the DevOps / CD pipeline

•  Virtualization requires: •  Various deployment models

•  Local, Server, Docker, Cloud

•  Ready to use transports and payloads •  Extensibility where needed •  Dynamic reconfigurable (provisioning) •  Scalable to use in performance test •  Integration in CD pipeline (APIs)

Q & A

•  Thank you