the gitb testing framework

THE GITB TESTING FRAMEWORK

Jacques Durand, Fujitsu America | [email protected] 1, 2011

GITB | http://www.ebusiness-testbed.eu

http://www.ebusiness-testbed.eu/

Overview

Objectives of a “Global” e-Business Test Bed

Testing Scenarios Aspects of a Testing Framework

2

Some Objectives

Promote and Facilitate Adoption of eBusiness Standards Testing is key to standards quality & adoption Global standards need a Global Test

Platform / Practices A Common & Global Testing

infrastructure for eBusiness Standards Addressing the needs of various stakeholders Designed and used/reused globally and

across industries

3

Testing the e-Business Stack

Messaging protocol

Meta

-Data

:In

terfa

ces, A

gre

em

en

ts

Business Process/ Exchange Choreography.

Business DocumentsM

essa

ges

Combination of standards / profiles Integrated testing

4

Test Bed Objectives

A Test Bed platform agnostic to eBusiness standards Extensible: Plug-in Components

Test suite Engine(s) , Message Adapters (AS4, ebMS2, etc.), Document Validators

Portable Test Artifacts across Test Beds Standardized Test Suites, Configurations, Test Logs … Test Registry/Repository archive / publish /

download Versatile Test Bed

Test Bed can simulate a business party (interaction) Or just analyze captured exchanges (validation) Both for Conformance and Interoperability Testing

5

Overview



6

GITB-compliant Test bed

“Validating” Test Suite

Document

ValidatorDocumen

tValidator

Test Case

Document

AssertionSet (DAS)

Test Registry / Repository

Query existingTest Cases

Create / DeployNew Test Suite

Test Scenario 1: Document Conformance•Creating & Using a “Validating” Conformance Test Suite for Structural & Semantic Validation of Business Documents•Human-driven

Test Designer

Get test ReportBusiness

Documents

• Use Test Suite• Upload Documents

Test Participant

Serv

ice

Inte

rface

Serv

ice

Inte

rface

7


Document

Validator

Document

Validator

ValidatingTest Case

Create / DeployInteracting Test Suite

• Execute• Get Conformance Test Report

Test Scenario 2: Document + Message Conformance•Creating & Using an “Interacting / Validating” Conformance Test Suite [Messaging protocol + QoS + Document]•Reusing the “Validating” Suite of Scenario #1SUT

Test Case[SUT] Interacting& Validating

Test Suite

ValidatingTest SuiteForDocumentValidation

GITB Test Repository

Serv

ice

Inte

rface

8


Document

Validator

Document

Validator

ValidatingTest Case


Get ConformanceTest Report

Test Scenario 2 Variant (a): delegating to another Test Bed

SUT

Test CaseInteracting /Validating Test Suite

ValidatingTest SuiteForDocumentValidation

GITB Test Repository


Delegates DocumentValidation to

Serv

ice

Inte

rface

ServiceInterface

9

Test bed

TestExecutionLog

TestReport

Test bed

drive

capture

2-Phase Testing (or pipe-line)

Test Scenario 2 Variant (b): separating the SUT interaction from the analysis (validation)

InteractingTest Suite

ValidatingTest Suite

SUT

10



• Register• Get Status• Get Notified• File Issues

Test Scenario 3: Document + Message + Process Interoperability Creating & Using an Interoperability Test Suite (driving several SUTs)

SUT 1

Interacting /validating Test Suite

ValidatingTest Suite

SUT 2• Drive• Capture

• Drive• Capture

Serv

ice

Inte

rface

• Register• Get Status• Get Notified• File Issues

11

Overview



12

Test Registry

& Repository

RemoteTest Bed /Test Agent

SystemUnderTest

SystemUnderTest

GITB Test Bed

Enable

Implement / process Manag

e

Test DesignerTest ManagerTest

Participant

Test Services- Test Design- Test Deployment- Test Execution- Test Repository

Test Artifacts- Test Logic (T.Suite…)- Test Configurations- Test Outputs (report..)

Test Bed Components- User-facing components- Core Platform components- Test Capability components

Control

Testing Framework in Action

13

Three Major Categories of Test Artifacts Test Logic Artifacts (define testing logic)

Test Suites/Cases, Document Assertions…

Test Configuration Artifacts (parameters and bindings for Test Logic) Mesg Adapters, Timing/Tuning, Message samples…

Test Output Artifacts (products of a Test run) Test Report, Test Logs

Depend on eB.

Standards

Depend on User

Environment

Depend on SUT

behavior

14

Test CaseTest CaseTest Case

Test Suite

A logical test unit to verify :- A business document- A business transaction- A message

Document

Validator

Document

Assertion

SetA set of processable requirements on a document:- schema- rules- dependencies

Test Logic Artifacts

Test Configurati

on

15

Test Artifact Standardization

Standardizing Test Artifacts: how far? Not enough lack of portability and reusability Too much too rigid to evolve, may not fit all tests

Level 1: Only standardize the header (metadata) Allows for storage, tagging and search.

Level 2: L1 plus some integration features: References or interfaces to other artifacts /

capabilities Container elements (e.g. message wrapper in a

log) Level 3: Whole content standardization

e.g. XML schema.

16

Test Services

Standardized Access Hide Test Bed specifics (e.g. legacy test

beds)

Serv

ice

Inte

rface

s

TestBed A

Test Designer

Test Participant

Serv

ice

Inte

rface

s

TestBed BTest Repositor

y Serv

ice

Inte

rface

s

17

Serv

ice

Inte

rface

s

Serv

ice

Inte

rface

s

TestBed BTestBed A

Test Participant

Test Services

Reuse of Testing Capabilities (e.g. Test Beds)

Test Suite HL7Transactions +messaging

Test Suite HL7 documentvalidation

SystemUnder Test

Delegation of document validation

18

Test Services

Test Design service: to manage Test Artifacts lifecycle in a Test Bed Create/Update/Delete local to a testBed.

Test Deployment service: to prepare for execution: Add Configuration to Test logic (bindings)

Test Execution service: to control all operations of a Test Bed related to Test execution.

Test Repository service: to archive, publish, discover Test Artifacts.

19

Test Bed Core Platform

Test Deploymen

tMgrTest

OperationMgr

Test Artifacts Persistence Mgr

Test AgentClient

RepositoryClient

Test Control

Mgr

Test

Suit

e

Inst

anti

ato

rUser-facingComponents

Test

Suit

e/C

ase

Edit

or

Managem

en

t C

onso

leTest CapabilityComponents (plug-ins)

Messaging

Adapter

DocumentValidatorDocumentValidator

Test SuiteEngineTest Suite

Engine

Test Deployme

ntService

Test DesignService

Test Executio

nService

Test RegRepService Registry/Repository Test bed

Test Bed Architecture

20

GITB Compliance Levels

How does a Test Bed comply with GITB recommendations?

GITB-Framework Compliance Test Bed designed in compliance with GITB

recommendations Test Services interfaces, Test Artifacts, allow for

plug-in Components

GITB-Service Compliance Adding [GITB] Test Service interfaces to “Legacy”

Test Beds Produce [GITB] Test Reports

21

the gitb testing framework

Documents

cen ws gitb2 plenary

validating suite of

interoperability testing

ebusiness standards

business party interactionor

document conformancecreating

pipelinetest scenario

ebusiness standardsaddressing