eurostar webinar testing cloud services kees blokland

Welcome to the 2013

www.eurostarconferences.com

This webinar is presented by Kees Blokland and he will discuss:

‘Going Cloudy? How to test SaaS?’

Join us on Twitter! @esconfs #esconfs

Going Cloudy?How to test SaaS?

Kees BloklandPolteq Test Services BV

The Netherlands

3

ISBN 978-1-937538-38-5

searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, talking, designing, making, verifying, measuring

6

EmailSurfTransfer

Develop and Test

Operate and Manage Store

7

EmailSurfTransfer

Develop and Test

Operate and Manage Store

storage claim

80% unused

redundancy, proliferation limitations

environmentally unfriendly

8

EmailSurfTransfer

Develop and Test

Operate and Manage Store

standard software bandwidth

internet technology

virtualization

SOA

9

searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, denying, talking, designing, making, verifying, measuring

EmailSurfTransfer

Develop and Test

Operate and Manage Store

US: National Institute of Standards and Technologyhttp://www.nist.gov

Essential characteristics

On-demand service

Broad network access

Resource pooling

Rapid elasticity

Measured service

Deployment models– private cloud– community cloud– public cloud– hybrid cloud

Service Models

Software as a Service

Platform as a Service

Infrastructure as a Service

Security

Continuity

Costs

Integration

Performance

Privacy

Testability

Multi platform Migration

Legislation

Vendor lock-in

Location

B.Y.O.D.

Cyber crime

User experience

Elasticity

Offline vs Online

Backup & recovery

Energy saving

Anywhere, anytimeMulti vendor

HostingImpact organisation

Standards

143

Security

Continuïty

Costs

Integration

Performance

Privacy

Testability

Multi platform Migration

Legislation

Vendor lock-in

Location

B.Y.O.D.

Cyber crime

User experience

Elasticity

Offline vs Online

Backup & recovery

Energy saving

Anywhere, anytimeMulti vendor

HostingImpact organisation

StandardsPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

14

Other customers

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

YOUROperational Profile

YOUROperational Profile

PLUS

15

Everything over the web

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

The idea:“it’s safe”

Home ground for hackers

16

Bring Your Own Device

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

No free choice of device.

No control.

17

Updates, patches, fixes, …

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

Planned and controlled

Do I have a choice?

18

Where is my data?

And is that OK?

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

In house.

Somewhere…

19

Performance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

20

Testing?

Check

Audit

IntakeSimulation

Inspections

Review

Assessment

Trial

Validate

Interview

Proof of concept

Test

Shadow run

……

21

Testing!

Check

Audit

IntakeSimulation

Inspections

Review

Assessment

Trial

Validate

Interview

Proof of concept

Test

Shadow run

……

TestenAuditReview

SimulatieInspectiesProef

AssessmentIntakeToetsen

InterviewProof of conceptControleSchaduwdraaien……

22

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

TestenAuditReview

SimulatieInspectiesProef

AssessmentIntakeToetsen

InterviewProof of conceptControleSchaduwdraaien……

23

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

24

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

25

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

26

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

Architecture

From “individual” risksto

“individual” test measures

27

Selection

Implementation

Production

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

28

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

29

Selection Criteria

Proof of Concept

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

30

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

31

Known measurestuned and tweaked

New measures developed

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

32

Load Testing

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

YOUROperational Profile

YOUROperational Profile

PLUSACTUAL MOMENT

33

Elasticity

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

Load and stress.

Load and elasticity.

34

load

load test – ‘up’

extend?

200 charged

100charged

no

yes

path test

5-4

99

100

101

boundary values‘up’tc 1: use=99, pay 100tc 2: use=100, pay 100tc 3: use=101, pay 200‘down’tc1: use=101, pay 200tc2: use=100, pay 100tc3: use=99, pay 100

boundary values

load test – ‘down’

35

Any device – any platform

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

Multiplatform testing.

Multiplatform testing.

36

3997 distinct Android devices

http://opensignal.com/reports/fragmentation.php

375-18

Internet Explorer 7

Internet Explorer 8

Firefox 3.6

Firefox 4

Safari 4

Safari 5

Chrome11

Opera11

Windows XP

Windows Vista

Windows 7

Windows 2003 Server

Windows 8

Windows CE

Linux

Unix

Mac OS Lion

Mac OS Snow Leopard

iOS

Android

Operating systems

Browsers

Multi-platform testing

Devices

Computer

Mobile phones

Tablet

PC

Macintosh

SUN

NOKIA …

Samsung …

Windows Mobile

iPhone ...

MOTOROLA …

Blackberry …

ASUS ...

38

Legislation + Regulations

=

Test basis

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

Incidental testing.

Compliancy testing.

39

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

40

ContinuousEnd-to-End Test

Functionalsand

non-functionals

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selection

Security

Continuity

Costs

Integration

Performance

Privacy

Testability

Multi platform Migration

Legislation

Vendor lock-in

Location

B.Y.O.D.

Cyber crime

User experience

Elasticity

Offline vs Online

Backup & recovery

Energy saving

Anywhere, anytimeMulti vendor

HostingImpact organisation

Standards

Check

Audit

Review

Simulation

Inspections

Intake

Assessment

Trial

Validate

Interview

Proof of concept

Test

Shadow run

……

42

Performance testing

Security testing

Testing for manageability

Availability and continuity testing

Functional testing

Migration testing

Testing caused bylegislation and regulations

Testing in production

Testing during selectionPerformance risks

Security risks

Availability and continuity risks

Functionality risks

Maintainability risks

Legislation and regulations risks

Suppliers and outsourcing risks

Architecture

From “individual” risksto

“individual” test measures

Testing starts early: in selection

Scope of testing is widened

Testing continues in production

43

ISBN 978-1-937538-38-5

Thanks for listening!

@esconfs#esconfs www.eurostarconferences.com

Go to www.eurostarconferences.com for more information