virtualizing the sme with opaque data processing

ca Devcenter

Virtualizing the SME withOpaque Data Processing Stefana Muller

DCX05S @StefanaMuller

CA TechnologiesSenior Principal Product Manager

2 © 2014 CA. ALL RIGHTS RESERVED.

For Informational Purposes Only

© 2014 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

This presentation provided at CA World 2014 is intended for information purposes only and does not form any type of warranty. Some of the specific slides with customer references relate to customer's specific use and experience of CA products and solutions so actual results may vary.

Terms of this Presentation


Any sufficiently advanced

technology is indistinguishable

from magic.

- Arthur C. Clarke


Abstract

Opaque Data Processing (ODP) uses patented algorithms to automatically find the relationships inside of nearly any data source, radically reducing the time required to create virtual services. Machine learning AI means the more data provided, the stronger the tool becomes. ODP makes moot the question, “Do you support that protocol?”

Stefana MullerCA Technologies

Senior Principal Product Manager


Agenda

SERVICE VIRTUALIZATION TODAY

CHALLENGES IN UNDERSTANDING DATA PROTOCOLS

ODP IN ACTION

OPAQUE DATA PROCESSING ESSENTIALS

OPAQUE DATA PROCESSING APPROACH

INCREASING ACCURACY OF ODP

1

2

3

4

5

6


The Big Problem - Constraints

“I can’t do anything until I have everything…and I never have everything!”

ESB

!

!

! ! !

System Unavailable

x x x

x

Invalid data Access Fees

Incomplete Development


How Service Virtualization Helps

“I have everything I need, when I need it!"

ESB

! ! !System Unavailable Invalid data Access Fees

Incomplete Development


Service Virtualization: How does it work?

CAPTURE PROCESS MODEL

Data Protocol Detection

Identification of Operation vs. other data in transaction

Identification of Magic Strings/Dates

De-Identify…

Record traffic between existing systems

Create from engineering specifications

From sources such as log files, sample data, packet capture and application insight files

Living, breathing “live” model

Sophisticated, contextual behavior

Automatic handling for dynamic properties


Service Virtualization: How does it work?

CAPTURE PROCESS MODEL

Record traffic between existing systems

Create from engineering specifications

From sources such as log files, sample data, packet capture and application insight files

Living, breathing “live” model

Sophisticated, contextual behavior

Automatic handling for dynamic properties

The DPHChallenge


The DPH Challenge

Must know the protocol‒ What if it is a proprietary protocol and the SME is no

longer available?

‒ What if the protocol is new, and there are no SMEs?

‒ What if you have a copybook and it is incomplete?

Must be able to identify the operation, body and syntax

It takes time to add new protocol support


ODP Essentials

High accuracy on the 4 well known protocols tested including IMS, LDAP, SOAP and Twitter (REST)

Speed and accuracy with Entropy Weighting + Message Clustering

The more data a service observes the more intelligent it becomes

Collaboration project between CA Labs, Swinburne University of Technology in Australia and the

CA Service Virtualization engineering and product teams

BRINGS TRUE ARTIFICIAL INTELLIGENCE (AI) TO SERVICE VIRTUALIZATION

HIGH ACCURACY: 99.6 – 100%

Virtualizes services without requiring any knowledge or decoding of the service protocols

Applies a genome sequence alignment algorithm, discovers byte-level patterns in message protocols

Virtualizes a much wider range of protocols without requiring a new DPH


ODP Approach

Current Approach

Opaque Data Processing

OPAQUE VSE

raw bytesrequest

responseCLIENT

machine learning match

VSE

Parse format delimit

request

responseCLIENT

matchdecode


ODP Approach – Byte-Level Alignment

Receive incoming request from client

Apply byte-level alignment to transactions in recording to find closest matching request (Needleman-Wunsch algorithm)

Perform byte-string substitution in matching response

Send modified response


Entropy Weighting

Increases accuracy of request matching

Weighs similar strings in a message higher than dissimilar strings

Assumption: The “Type”/Operation of a request is more important to match before the record information itself

For example: AddUser: 123456789012345

“Add User” – 7 bytes/higher priority“123456789012345” - 15 bytes/lower priority


ODP Approach with Entropy Weighting

ClientOpaque

ResponseGenerator

TransactionLibrary

StringSubstitution

Request

GeneratedResponse

Request

MatchingResponse

Matching Response

Modified Response

Needleman-WunschMatch

ClientOpaque

ResponseGenerator

TransactionLibrary

StringSubstitution

Request

GeneratedResponse

Request

MatchingResponse

Matching Response

Modified Response

CalculateEntropyWeighing

WeightedNeedleman-WunschMatch


ODP Accuracy with Entropy Weights

ProtocolSimple Method

% CorrectEntropy Weights

% Correct Best Entropy Measure

IMS 77.4 100 Richness

CICS 100 100 Shannon/Richness

LDAP ASN.1 94.2 94.6 Shannon

LDAP Text 100 100 Shannon/Richness

Twitter (REST) 99.5 99.6 Shannon

SOAP 100 100 Any


When to Apply ODP

ODP provides a turn key solution to virtualize protocols otherwise not supported by CA Service Virtualization

When you need reasonable data and you do not need to force specific behavior

For encrypted messages, decryption needs to occur before ODP analysis

ODP currently treats all transactions as stateless

?


Q & A


Credits

to the team of engineers and researchers for their hard work and determination in coming up with this innovative way to address the ever-growing challenge of protocol support.

Miao Du, Jean-Guy Schneider, Jun Han and John Grundy – Faculty of Information and Communication Technologies, Swinburne University of Technology, Australia

Steve Versteeg – Research Staff Member and Team Lead at CA Labs

Robert Williams (Staff Software Engineer and Architect) and Chris Kraus (Sr. Principal Product Manager) from the CA Service Virtualization product team

Dr. Steve VersteegCA Labs

Ms. Miao DuPhD Candidate

Dr. Jean-Guy Schneider

Prof. John Grundy

Prof. Jun Han

Robert WilliamsStaff Software Engineer,

CA Technologies

Chris KrausSr. Principal Product Manager

CA Technologies

Thank you . . .


Session Evaluation

Please provide your feedback about this session

Session Name:Virtualizing the SME with Opaque Data Processing

Access inside the CA World Mobile App

Click on SURVEY/SESSION EVALUATION If your badge was scanned at the entrance to

this session, click on the name of this session.


Devcenter CA Continuous Application Insight

CA Technologies

Devcenter CA Application Test

CA Technologies

Related Technologies

DevcenterCA Service Virtualization

CA Technologies

Devcenter CA Service Virtualization for SAP

CA Technologies

virtualizing the sme with opaque data processing

Software

sample data

data source

new protocol support

unknown protocol

proprietary protocol

service virtualization

existing systemscreate

existing services