eurostar webinar testing cloud services kees blokland
DESCRIPTION
On September 17 Polteq contributed to the EuroSTAR online event ”Software Testing Summit” with a webinar about testing cloud services with title ”Going Cloudy? How to test SaaS?”TRANSCRIPT
Welcome to the 2013
www.eurostarconferences.com
This webinar is presented by Kees Blokland and he will discuss:
‘Going Cloudy? How to test SaaS?’
Join us on Twitter! @esconfs #esconfs
Going Cloudy?How to test SaaS?
Kees BloklandPolteq Test Services BV
The Netherlands
3
ISBN 978-1-937538-38-5
searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, talking, designing, making, verifying, measuring
6
EmailSurfTransfer
Develop and Test
Operate and Manage Store
7
EmailSurfTransfer
Develop and Test
Operate and Manage Store
storage claim
80% unused
redundancy, proliferation limitations
environmentally unfriendly
8
EmailSurfTransfer
Develop and Test
Operate and Manage Store
standard software bandwidth
internet technology
virtualization
SOA
9
searching, recording, accounting, paying, writing, reviewing, tracking, calculating, developing, listening, analyzing, transmitting, learning, controlling, purchasing, testing, alarming, changing, updating, deleting, accessing, rejecting, correcting, studying, booking, receiving, tracing, protecting, deciding, managing, teaching, facilitating, identifying, copying, removing, demonstrating, checking, showing, selecting, subscribing, unsubscribing, sharing, mailing, communicating, reading, playing, working, meeting, gambling, shopping, storing, cross checking, retrieving, configuring, sketching, saving, accelerating, enhancing, creating, growing, checking in, checking out, finding out, reaching, denying, talking, designing, making, verifying, measuring
EmailSurfTransfer
Develop and Test
Operate and Manage Store
US: National Institute of Standards and Technologyhttp://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Rapid elasticity
Measured service
Deployment models– private cloud– community cloud– public cloud– hybrid cloud
Service Models
Software as a Service
Platform as a Service
Infrastructure as a Service
Security
Continuity
Costs
Integration
Performance
Privacy
Testability
Multi platform Migration
Legislation
Vendor lock-in
Location
B.Y.O.D.
Cyber crime
User experience
Elasticity
Offline vs Online
Backup & recovery
Energy saving
Anywhere, anytimeMulti vendor
HostingImpact organisation
Standards
143
Security
Continuïty
Costs
Integration
Performance
Privacy
Testability
Multi platform Migration
Legislation
Vendor lock-in
Location
B.Y.O.D.
Cyber crime
User experience
Elasticity
Offline vs Online
Backup & recovery
Energy saving
Anywhere, anytimeMulti vendor
HostingImpact organisation
StandardsPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
14
Other customers
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
YOUROperational Profile
YOUROperational Profile
PLUS
15
Everything over the web
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
The idea:“it’s safe”
Home ground for hackers
16
Bring Your Own Device
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
No free choice of device.
No control.
17
Updates, patches, fixes, …
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
Planned and controlled
Do I have a choice?
18
Where is my data?
And is that OK?
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
In house.
Somewhere…
19
Performance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
20
Testing?
Check
Audit
IntakeSimulation
Inspections
Review
Assessment
Trial
Validate
Interview
Proof of concept
Test
Shadow run
……
21
Testing!
Check
Audit
IntakeSimulation
Inspections
Review
Assessment
Trial
Validate
Interview
Proof of concept
Test
Shadow run
……
TestenAuditReview
SimulatieInspectiesProef
AssessmentIntakeToetsen
InterviewProof of conceptControleSchaduwdraaien……
22
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
TestenAuditReview
SimulatieInspectiesProef
AssessmentIntakeToetsen
InterviewProof of conceptControleSchaduwdraaien……
23
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
24
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
25
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
26
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
Architecture
From “individual” risksto
“individual” test measures
27
Selection
Implementation
Production
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
28
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
29
Selection Criteria
Proof of Concept
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
30
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
31
Known measurestuned and tweaked
New measures developed
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
32
Load Testing
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
YOUROperational Profile
YOUROperational Profile
PLUSACTUAL MOMENT
33
Elasticity
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
Load and stress.
Load and elasticity.
34
load
load test – ‘up’
extend?
200 charged
100charged
no
yes
path test
5-4
99
100
101
boundary values‘up’tc 1: use=99, pay 100tc 2: use=100, pay 100tc 3: use=101, pay 200‘down’tc1: use=101, pay 200tc2: use=100, pay 100tc3: use=99, pay 100
boundary values
load test – ‘down’
35
Any device – any platform
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
Multiplatform testing.
Multiplatform testing.
36
3997 distinct Android devices
http://opensignal.com/reports/fragmentation.php
375-18
Internet Explorer 7
Internet Explorer 8
Firefox 3.6
Firefox 4
Safari 4
Safari 5
Chrome11
Opera11
Windows XP
Windows Vista
Windows 7
Windows 2003 Server
Windows 8
Windows CE
Linux
Unix
Mac OS Lion
Mac OS Snow Leopard
iOS
Android
Operating systems
Browsers
Multi-platform testing
Devices
Computer
Mobile phones
Tablet
PC
Macintosh
SUN
NOKIA …
Samsung …
Windows Mobile
iPhone ...
MOTOROLA …
Blackberry …
ASUS ...
38
Legislation + Regulations
=
Test basis
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
Incidental testing.
Compliancy testing.
39
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
40
ContinuousEnd-to-End Test
Functionalsand
non-functionals
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selection
Security
Continuity
Costs
Integration
Performance
Privacy
Testability
Multi platform Migration
Legislation
Vendor lock-in
Location
B.Y.O.D.
Cyber crime
User experience
Elasticity
Offline vs Online
Backup & recovery
Energy saving
Anywhere, anytimeMulti vendor
HostingImpact organisation
Standards
Check
Audit
Review
Simulation
Inspections
Intake
Assessment
Trial
Validate
Interview
Proof of concept
Test
Shadow run
……
42
Performance testing
Security testing
Testing for manageability
Availability and continuity testing
Functional testing
Migration testing
Testing caused bylegislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity risks
Functionality risks
Maintainability risks
Legislation and regulations risks
Suppliers and outsourcing risks
Architecture
From “individual” risksto
“individual” test measures
Testing starts early: in selection
Scope of testing is widened
Testing continues in production
43
ISBN 978-1-937538-38-5
Thanks for listening!
@esconfs#esconfs www.eurostarconferences.com
Go to www.eurostarconferences.com for more information