smarter infrastructure testing: a service oriented approachict infrastructure deployment new ict...

Smarter Infrastructure

Testing: A Service Oriented

Approach

Egbert Bouman

Maintain

Smart Infrastructure TestingA service-oriented approach

Egbert Bouman, Maintain B.V.

Ellen Zondag, Kadaster

The Netherlands

Complex ICT infrastructures

FELETFELET

CEDAR

GL

CEDAR

GL KASKAS DelfiDelfi

CEDAR

AR/AP

CEDAR

AR/AP

IDITIDIT

SVSSVS

Postcode

Tabel

Postcode

Tabel

MAREGMAREG

Document

Flow

Services

Document

Flow

Services

DOC1

Generate

DOC1

Generate

Meeting PointMeeting Point

TP

Admini-

stratie

TP

Admini-

stratie

CervisCervis

POR/

ADN

POR/

ADN

ADN

Postbus

ADN

Postbus

PMB Polis Mutatie Bericht

PPR Prolongatie Bericht

PBI Polis Interface Bericht

CetisCetis

= Interface

xxx = Interface identificatie

Legenda

AUDA

SCAN

(RDC)

AUDA

SCAN

(RDC)

BOA

ARBO

BOA

ARBO

SEOSSEOS

Proces

Mngmnt

Proces

Mngmnt

KvKKvK

ARBO

Totaal

ARBO

Totaal

FISHFISH

BCS

BOA -

ARBO

WN-

bestand

Klant Order

Schermen

Klant Order

Schermen

Content

Manage-

ment

Content

Manage-

ment

E-ABSE-ABS

Scan

Straat +

FAX +

email

Scan

Straat +

FAX +

email

Content

Manage

ment

Content

Manage

ment

Klant

Beeld

Klant

Beeld

Contact

Historie

Contact

Historie

TRAVONTRAVON

DOC1

Inter-

active

DOC1

Inter-

active

002 001

007

008

009

010

011

012

013

021

022

023

024

025

077

027

028

030

032

033

035

037 038039

040

041

042 043 045 046

052

053

057

058

059

060

xxx = Bestandsinterface

BARBAR

065

BETA

(AAB)

BETA

(AAB)

MultiFit

Desktop

MultiFit

Desktop

066

GIM ServerGIM Server069

GIM

Server

GIM

Server070

SLBSLB

074

RDW

Services

RDW

Services

026

078

101

WALVIS

UWV

WALVIS

UWV

080

081083

084

085050

LDAPLDAP087086

049

DDDDDD

Overvoer

Volmacht

089

090

091

Batch

Adapter

Batch

Adapter

Mutatie

Net

Mutatie

Net

PP

Offerte

tool

PP

Offerte

tool

092

Klant

Order

Scherm

en

Klant

Order

Scherm

en

GIM

Ser-

ver

GIM

Ser-

ver

Inter-

face

Man

ager

Inter-

face

Man

ager

= Applicatie076

003093

094

100

004

= Schermpresentatie097

Printstraat

+ FAX +

e-mail

Printstraat

+ FAX +

e-mail

098

UWV031

VARVARClearing-

huis

Regres

Clearing-

huis

Regres

FIOD -

TAPE

099

Dossier

Volg

Systeem

Dossier

Volg

Systeem

= Request-Reply

= Fire&Forget

= Batch

= R&R en F&F

en Batch of nog

niet bepaald

102

103

ICT Infrastructure Deployment

New ICT infrastructure components Such as: OS‟s, DBMS‟s, network, storage solutions

Make end user and business applications work

Must be seamlessly “fitted in” With uninterrupted support of business app‟s and processes

Maintaining agreed service levels

Smooth and fast Excessive human resource claims not acceptable

Neither a top heavy and lengthy process

A universal challenge!

Primary goals are about minimizing risks : Keep faulty components away from production environment

Ensure reliability, availability, supportability (RAS)

Check unharmed application functionality and availability

Secundary goals : Enable ICT governance

Embed into maintenance standards and procedures

Prepare and train maintenance staff

Testing Infrastructure Deployment

Testing plays a vital role in this game

Different from application testing

Different Quality aspects Less: functionality, usability

More: reliability, availability, supportability, scalability, …

Different Stakeholders Less: business stakeholders, end users

More: acceptance by maintenance organisation

Different Components Less: custom built

More: standard components, commodity, COTS, open source

Different Objectives Less: realising competitive edge

More: cost reduction, standardisation

Fairly different test scope and risks

State of the practice

Testing methods, models and techniques Optimized for application testing

Not for infrastructure testing

Dedicated test types (PAT, Exploitation test) a technician‟s party

Testing partly managed by outsourcing partners Technical management: often OK

Application management: maybe OK

Functional management: maybe not OK

Some help available from: ITIL, ASL

Not from the testing communityDedicated test methods hardly available

Dutch Land Register

In Dutch: “Het Kadaster” Since 1832

Main office in Apeldoorn (near Utrecht)

Semi-government, 2000 employees, 400 in IT

Highly complex ICT infrastructure: Geographical and administrative applications,

SAP, Windows, VMS, Unix, Citrix, Mainframe

Crucial data for past and current society

Kadaster online (www.kadaster.nl)

e-Europe Awards for eGovernment 2005

http://www.kadaster.nl/index.html

Kadaster ICT environment

Kadaster TSC

Professional „Test Support Centre‟ Foundations laid in 1993

Full fledged independent department since 1999

Valued partner Good track record

Sound methodology

Testing business applications

Professional staff 35 test professionals

Manager: Jan van der Horst

Kadaster infrastructure test issues

Major business disruptions

Too little, too late, too long

No adequate process, test types and milestones

Technical know-how hardly shared

Miscommunication between testers and …

Stakeholdership not addressed

Testing optimized for business app’s only

Ap

pli

cati

on

Infr

astr

uctu

re

The “black hole” before PPT

ST GFT AF

S

PP

T

Reali

sati

eR

ealisati

on

?

Develop a dedicated test approach… … in close co-operation with Kadaster TSC…

… to deliver a reliable quality report…

… concerning all applicable quality aspects…

… for new infrastructure components…

… in combination with business applications…

… in order to fit them into place more smoothly…

… in a shorter time…

… and without disruptions

And let‟s put it into place … … starting with a pilot project…

… in the Kadaster organisation

Commission to Maintain

Who is Maintain?

Maintain Independent testers

For 16 years

Top-200 companies

70+ consultants

Mainly in the Netherlands

Recently: Valori group founded

SmarTEST test approach Just enough, smarter testing

Flexibility and adaptability

Best practices

Eurostar sponsor

www.maintain.nl

The answer

A dedicated test process framework For fitting infrastructure components into place

Jointly developed by Kadaster and Maintain

SmarTEST based

We call it FitTEST

Four key principles

Key principles

1. Get to know your infrastructureand adopt a service-oriented reference model, i.e. TOGAF

2. Define service-specific best test practicesand make „how to test‟ guidelines for each service type

3. Think parallel rather than linearand adopt a non-phased test approach

4. Apply a SMART artifact hierarchyand optimize it for re-use and knowledge sharing

Standard software Custom build software

Communication Infrastructure

Native

Manage

ment T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er

Service

s

Servers and Storage

Operating Systems

Mid

dlew

areM

anagem

ent

Gene

ric Manage

ment

Tools

Mainframe

Quality

aspects

Test

objects

Specific Fit Criteria

PRIMA risk matrix

Generic

GuidelinesService-specific, with tips and

tricks

Project

documentation:

reqts and specs

Test DesignTest scenarios, clusters, cases

FitTEST ManualOverall handbook Infrastructure

Testing

Direct

stakeholder

input

Validation,

completeness check

Generic Fit criteriaAll stakeholders

Quality ProcessKadaster Quality Handbook

Get to know your infrastructure

Situation at Kadaster Three different inventories available

No clear grouping or reference model

Need for structure and grip From Kadaster service philosophy viewpoint

Why not an existing standard?

Architecture Frameworks

Which one to adopt?

Architecture Frameworks

Popular Architecture Frameworks in Europe: Zachman Developed at IBM

TOGAF The Open Group Architecture Framework (Open Group)

IAF Integrated Architecture Framework (CGEY)

We selected TOGAF because: It‟s service-oriented, it‟s open and it‟s free

It provides a suitable Technical Reference Model

Further reading: www.opengroup.org/togaf

www.software.org/pub/architecture

(enumeration and discussion of many available frameworks)

http://www.opengroup.org/togaf

http://www.software.org/pub/architecture

TOGAF technical reference model

Highest level:

three layers

Two bottom layers

represent infrastructure

Top layer out of scope

Applications

Applicationsplatform

Communication infrastructure

API: Applicatie Platform Interface

CII: Communicatie Infrastructuur Interface

www.opengroup.org/togaf

http://www.opengroup.org/togaf

The Kadaster infra-model

Standard software Custom built software


Native

M

anagem

ent T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er S

ervice

s

Servers and Storage

Operating SystemsM

iddle

ware

M

anagem

ent

Generic M

anagem

ent

Tools Infra

Structure

Main frame

Business Applications

Benefits of TOGAF for testing

The benefits of this open, service-oriented reference

model (TOGAF) for testing:

Standard Test suites available Free, proven

Support for development of additional suites

Easy to fit into currently popular service approach: Enterprise Service Bus

Software Oriented Architectures

XML, SOAP testing

Key principles







Native

Manage

ment T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er

Service

s

Servers and Storage

Operating Systems

Mid

dlew

areM

anagem

ent

Gene

ric Manage

ment

Tools

Mainframe

Quality

aspects

Test

objects


PRIMA risk matrix

Generic


tricks

Project

documentation:

reqts and specs



Testing

Direct

stakeholder

input

Validation,

completeness check



Service-specific best test practices

Empower your skills They are there

Just find, document and deploy them to everyone

For each service: Make standard checklists and acceptance criteria (what to test)

Select and refine smart test techniques (how to test)

Add tips and tricks

Bundle in one handy guideline for each service group

Better in-company test support: „How to‟ reference manual and checklists per service group

Improved knowledge management and skills allocation

Bundled in “Test Guidelines”

Ownership and

continuous

improvement

Not obligatory But strongly

recommended

Based upon growing

experience

A typical guideline: OS’s

Introduction

Characteristics of OS‟s General

Open VMS

True64 UNIX

Windows

Test approach Component Integration Test

Application Integration Test

Management Integration Test

Applicable Test Techniques

Top-10 Quality Aspects

Hints for Test execution

Key principles







Native

Manage

ment T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er

Service

s

Servers and Storage

Operating Systems

Mid

dlew

areM

anagem

ent

Gene

ric Manage

ment

Tools

Mainframe

Quality

aspects

Test

objects


PRIMA risk matrix

Generic


tricks

Project

documentation:

reqts and specs



Testing

Direct

stakeholder

input

Validation,

completeness check



Test Types

Test types A well-known sequence: UT, IT, ST, AT

Test types for infrastructure testing: CT: Component Test

= intake and unit test

CIT: Component Integration Test

= does it fit into the existing technical infrastructure

AIT: Application Integration Test

= does it fit the business applications

MIT: Maintenance integration test

= does it fit into the maintenance organization

PPT: Pre Production Test

= last gateway before production, in production like environment

How to organize test types in time

CTCIT

AITMIT

PPTTiled

Parallel

CIT AIT MITSequentially CT PPT

CT PPT

Mix

Iterative

The Integration Circle or Fit Circle

Fit

Criteria

Intake

De

charge

To Production

Environment

CT

PPT

Ready

for fit

Key principles







Native

Manage

ment T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er

Service

s

Servers and Storage

Operating Systems

Mid

dlew

areM

anagem

ent

Gene

ric Manage

ment

Tools

Mainframe

Quality

aspects

Test

objects


PRIMA risk matrix

Generic


tricks

Project

documentation:

reqts and specs



Testing

Direct

stakeholder

input

Validation,

completeness check



The 5 principles of SMART testing

Strategic Connect to strategic business goals, critical

succes factors and KPI‟s

Men-oriented Plan and design your process for maximum

stakeholder participation and commitment

Adaptable Be prepared for change and opt for flexibility, just

enough process and documentation

Risk based No risk, no test

Transparant Create transparancy and report visually, based

upon fit and acceptance criteria

Basic Test Design Process

Quality

aspects

Test

objects


PRIMA risk matrix

Generic Project or maintenance release

Workshop with

stakeholders

Comment


tricks

Project

documentation:

reqts and specs

Facilitated bij Test

Support Centre


Risk-based selection ,

development and

(autom.) execution


Testing

Direct

stakeholder

input

Validation,

completeness check



Check Fit Crteria with

PRIMA matrix

Kadaster Pilot Project

Citrix Metaframe XP roll-out Complete re-arrangement of server farms

Integration with other management facilities (CA Unicenter TNG

framework, agents, …) and maintenance procedures

High risk for business application availability

All desktops affected

Successsfully tested On time, on schedule

Positive evaluation report

Management decision to

adopt FitTEST approach

Issues still being addressed

Technics Test environments don‟t meet the needs (yet)

So testing is still done too late on an unstable environment

Culture: Ongoing struggle to get technical testers to use the handbooks

The guidelines have to grow by using them

Improvement feedback loop still weak

More than (management) commitment

required

Achievements made

Process: Black-hole is filled, successful pilot

More „grip‟ to the testing process, improved TSC role

TOGAF based reference model adopted Maintained bij Architecture Management department

Handbooks available Overall “Handbook Infrastructure Testing”

Six guidelines for specific service-groups

Method incorporated into Kadaster practices Integration Test Circle incorporated

Growing use of acceptance criteria (not only infrastructure)

Kadaster improvements (being) achieved

So, now available to you:

A dedicated architectural reference model TOGAF, optimized for testing

Service-oriented, open, free

An effective process for infrastructure testing Fitting in components faster and more smoothly

By a combination of linear and non-linear process

Based upon the „integration test circle‟ concept

A dedicated design and documentation structure From risk analysis to test design

Optimized for re-use and knowledge sharing



Native

Manage

ment T

ools

Security S

ervice

s

DB

MS

‘s

Oth

er

Service

s

Servers and Storage

Operating Systems

Mid

dlew

areM

anagem

ent

Gene

ric Manage

ment

Tools

Mainframe

Quality

aspects

Test

objects


PRIMA risk matrix

Generic


tricks

Project

documentation:

reqts and specs



Testing

Direct

stakeholder

input

Validation,

completeness check



For the sake of less business disruptions

Thanks for your attention

Further questions, need help? Please feel free:

[email protected]

[email protected]

smarter infrastructure testing: a service oriented approachict infrastructure deployment new ict...

Documents