bcs - sigist - testing as a service - models
Post on 02-Aug-2015
409 Views
Preview:
TRANSCRIPT
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Better TaaS
5th June 2015
jonathon.wright@hitachiconsulting.com
Testing as a Service - ModelsJonathon Wright, Director, Testing, Quality, Assurance
2
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Hitachi - a global industry leader.Society Changes, Hitachi Transforms It.
$93.4
Billion947Companies
320,000+Employees
$3.4
Billion
Hitachi ranks 78th on the 2014 FORTUNE Global 500®.
© Hitachi Europe Ltd. 2014. All rights reserved.
3
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Hitachi – Digital Business in Europe
Information & Telecommunication Solutions Distribution Solutions
Railway Solutions Smart Cities
Service Opportunities
IT Service, Data Centre Solutions, Consulting
System Solutions
Large-scale systems, Highly reliable systems
Finger Vein Authentication
Worlds FastestElevator (40mph)
Hardware, Software
Rolling stock + maintenance
Electrical components
Signaling / Train control systems
Nuclear Power Business
Mining Equipment
Platforms
Smart eco cities
Big Data Infrastructure
Energy Consumption
@jonathon_wright
#BCS_SIGIST
Water & Natural Resources
Floating train 500kmph
4
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
What is BiModal? and how is FluidIT going to affect CoreIT?
4
“the creation of new business designs
that connect people, business and things
(physical objects that are active players
and contribute to business value)
to drive revenue and efficiency. ”Gartner ® 2014
@jonathon_wright
#BCS_SIGIST
5
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Digital Enterprise - BiModal
How is FluidIT going to affect CoreIT?
Fluid IT• Agile and fluid
• Innovation, brand,
profit measures
• Think continuous
• Think days, weeks
Core IT• Reliable,
compliant, secure
• Think price/performance
• Plan and approvals driven
• Long life-cycles
6
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Digital Enterprise is not just about “disrupters”
@jonathon_wright
#BCS_SIGIST
6
7
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014
Digital Transformation
Disrupters
Testing in the Digital Enterprise
8
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Gartner, 2014, ‘Building your Digital Enterprise’ Gartner event, Poland, 15th May 2014
Testing in a ‘Digital Enterprise’ Transformation Age
@jonathon_wright
#BCS_SIGISTTesting in the Digital Enterprise
@jonathon_wright
#BCS_SIGIST
‘Digitalised’ businesses are able to better adapt and exploit opportunity
Digital Enterprise Blueprint + Roadmap
9
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing industry is changing forever
Intel© XDK IoT in 2015
Does ‘Testing as a Function’ still have a future?
10
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Hitachi – Smart Cities - Copenhagen
Effecting Copenhagen Cleantech Cluster
City
Admin
Mobility
"Green and Comfortable
Transportation"
"Lower energy consumption"
- City of bikes
- ITS
- Mobility Management
- Copenhagen Connecting
- Buildings
- Transport
- Procurement
- Street Lighting
Energy
"Carbon-neutral city"
- Smart meter by 2020
- Wind turbines
- Biomass in power plant
- Smart Cities
@jonathon_wright
#BCS_SIGIST
Carbon Neutrality
by 2025
11
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing as a Function
Legacy Methodologies
Software Development Lifecycle
Hardware Analysis
Testing
Process
MethodologyBusiness
Software Development Lifecycle (SDLC) Model
12
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing industry is not ready
Support for business intelligence powered by testing ‘small data’
✔
Hitachi Content Platform Solution
During deployment of
the Hitachi Content
Platform Solution
70TB of satellite
information was
ingested into the
solution platform with
a daily growth rate of
50GB/day.
Business
Intelligence
Platforms
Business
Intelligence
Infrastructure
* http://www.hds.com/assets/pdf/hitachi-storage-solutions-at-work-nasa.pdf
13
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing as a Activity
Future Methodologies
Solution Delivery Lifecycle
Application
Portfolio
Business
IntegrationEnterprise
Management
Solution Delivery Lifecycle integration (SDLCi) Model
14
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing industry needs new models
Global challenges are not suitable for ‘Testing as a Function’ silos
Complex eco-
infrastructures
intelligent
machine-to-
machine inter-
connections
empowered by
‘Big Data’
* http://www.libelium.com/libelium-smart-world-infographic-smart-cities-internet-of-things/
15
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
New Model for Testing*
Model Based Testing (MBT)
*Paul Gerrard, 2014, ‘New Model for Testing’ discussion paper published, 6th August 2014
New Model for Testing
16
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
@jonathon_wright
#BCS_SIGISTTesting as a Service - Models
https://leanpub.com/taas
17
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Business Delivery Management
Model Based Design (MBD)
Testing as a Service - Models
Business Lifecycle Management
Problem
Business Abstraction Layer
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)
Business Delivery Management - ‘As A Service’ layer
Test Abstraction Layer – Test Solution Delivery
Business Delivery Management - ‘Testing as a Service’ layer
Portfolio Lifecycle Management
Solution Lifecycle Management
Application Lifecycle Management
Test Abstraction Layer – Test Solution Delivery
Solution
18
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
• Business Lifecycle Management
– Business on a Page (BoaP) level abstraction or encapsulation representing the core
businesses differentiators such as agility, visibility and flexibility
• Portfolio Lifecycle Management
– number of solutions that are represented by a logical grouping as part of a Business Portfolio
or split by Business Domain or Workstream
• Solution Lifecycle Management
– covers logical groupings of applications that make up the solution landscape
• Application Lifecycle Management
– refers to a single application instance
Business Delivery Management
Business Lifecycle Management
19
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Business Abstraction Layer
Business Abstraction Layer
Model Based Design (MBD)
Testing as a Service - Models
Business Solution Discovery Business Solution Delivery
Business
on a Page
Business Level
Answers
Business Level
Questions
Business Level
Stories
Business
Requirements
Business Process
Validation
Business Process
Modelling
Business Domain
Knowledge
Business Process
Intelligence
Business Process
Design
Business Process
Scenarios
Business Process
Tests
Business Process
Data
Business Process
Transactions
Business User
Stories
Business Level
Keywords
Business Non-
Functional (NFRs)
Business Epic
Themes
Business Definition Layer
Business
Acceptance Criteria
Business Definition
Language
Business
Acceptance Tests
Business Process
Reporting
Business Process
Behaviours
Business Process
Components
20
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Business Delivery Management
Model Based Design (MBD)
Business Abstraction Layer
Business Solution Discovery Business Solution Delivery
Business Definition Layer
Test Abstraction Layer
Test Solution Discovery Test Solution Delivery
Test Definition Layer
Problem
Business Delivery Management (‘As A Service’ Abstraction Layer)
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLi)
*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Testing as a Service - Models
Solution
Logistics
21
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Why ‘As A Service’
• Testing as a Service (TaaS)
• Software as a Service (SaaS)
• Platform as a Service (PaaS)
• Infrastructure as a Service (IaaS)
• Hardware as a Service (HaaS)
Testing in the Wild - Autonomous Vehicles
@jonathon_wright
#BCS_SIGIST *Jonathon Wright, Top Gear Track, Test Results: 1:47.63, http://youtu.be/P8JnQV0nXrc, 31st October 2014
Performance Test Results: 1:47.63
22
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Problem
Business
Customer
SolutionProblem
Definition
Business
Intelligence
(CIO, COO, CTO)
Solution
Decommission
Social
Intelligence
Business
Innovation
Social
Innovation
‘As A Service’ - Model
Model Based Design (MBD)
Testing as a Service - Models
Business Problem, Idea or Challenge (cost, benefit, time to market)
Business Solution Delivery Quality (cheap, good, fast)
‘As A Service’
CustomerEnd-Users
23
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011
Business Delivery Management
Service Abstraction Layer - Solution Delivery Lifecycle integration (SDLCi)
Testing as a Service - Models
Portfolio Lifecycle Management
Business Lifecycle Management
Solution Lifecycle ManagementBusiness Delivery Management
AdviceProblem
Business Solution
Discovery
Business
Solution DeliveryContinuous Delivery Continuous Deploymentvs.
Service Layer (Lean)
24
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014
Model Based Design
Testing as a Service - Models
Shift Left
Shift Up
Shift Down
Shift Right
Test
in
Dev
Business
Acceptance
Testing
Test
Ops
User
Acceptance
Testing
Social
Enterprise
Solution Delivery Lifecycle integration (SDLi)
Continuous, Integration, Build & Deployment (CIBD)
Business
Enterprise
(CIO, COO, CTO)
Testing as a Service - Model
Testing As A
Service
25
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Introduction to Model Based Design (MBD)
Testing as a Service - Models
Blueprint(s) Pattern(s) Recipe(s)
www.eurostarconferences.com* Test Automation Patterns, testautomationpatterns.wikispaces.com & testing-store.com
@jonathon_wright
#BCS_SIGIST
26
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Blueprint(s)
Testing as a Service - Models
We used drawing (visual 2D art)
to represent the foundation
of basic blueprints
GEN 1
GEN 1k
“Truly wonderful, the mind of a child is.”
This demonstrates that models created from
a single entity (transcendence) only represent
ontology of the domain of discourse and
exclude the quantitative intelligence of others.
“No matter their experience, masters can still
learn from their apprentices.“
“That’s No Moon!”
In Digital Enterprise
= Business on a Page
Enterprise Collaboration is the Evolution
(not revolution) of detailed expert Blueprints
27
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Blueprint(s) – Real World Example
SAFe™ 4.0
An example
Blueprint(s) could
be any process
template that
defines the
high level
flows
“The best aspects
of these proven
test approaches
demonstrate how
they have evolved
over the past
decade, and this
echoes some of
the changes
towards more lean
and agile business
methodologies.
They are in a
constant state of
evolution – just as
the underpinning
technology evolves
over time.”
28
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Pattern(s)
Testing as a Service - Models
Death Star Interior / Inner Chasm & Corridors
Collaboration of individual applies unique
patterns that will refine, predict and
inform the model through exploration
(enquiring and challenging) each other
Quantum Multifractality
Applying constructive patterns
from two or more point sources
(interference) to render multi-
dimensional (3D) models
GEN 2k
GEN 1080Each iteration can increase the
representation of the model by
applying and interpreting data
29
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Pattern(s) – Real World Example
Test Infrastructure as a Service (TIaaS.net) – TestOps 1.0 (VM Role Dispenser)
* Test Automation as a Service – Architecture, TAaaS.net or TestOps.net
“The best aspects
of these proven
test approaches
demonstrate how
they have evolved
over the past
decade, and this
echoes some of
the changes
towards more lean
and agile business
methodologies.
They are in a
constant state of
evolution – just as
the underpinning
technology evolves
over time.”
“Over the last five
years we have been
developing a azure
cloud based solution to
provide true
Test Automaton as a
Service using the
hybrid approach
specifically designed
with the Hybrid Cloud
(Private/Public) in
mind to utilise the
instant scalability,
flexibility and
availability of the
Cloud.”
30
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Recipe(s)
Testing as a Service - Models
Early recipes date back to
1600BC, IFTTT recipes
(conditional statements) =
DestroyDeathStar1.0I. All wings report in
II. Lock S-foils in attack position
III. Switch deflectors ON
IV. Keep visual scanning
V. Start attack run
VI. Move into position
VII. Stay in attack formation
VIII. Stay on target
IX. Use the FORCE
GEN 4k
• Channels are the basic
building blocks
• Triggers are the "this" part
of a recipe
• Actions are the “that” part
of the recipe
• Recipes are the
“equations” made from
Triggers and Actions
• Ingredients are the basic
data model, node, quantum
materials (dot) particles
or even atom(s)
Enterprise Gamification
represents the success criteria
e.g.
“Great shot kid that
was one in a million.”
31
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD) – Recipe(s) – Real World Example
Test Automation as a Service (TAaaS.net) – TestOps 1.0 (>5 Years)
Select the Business Process Tests Generated Recipe
“The best aspects
of these proven
test approaches
demonstrate how
they have evolved
over the past
decade, and this
echoes some of
the changes
towards more lean
and agile business
methodologies.
They are in a
constant state of
evolution – just as
the underpinning
technology evolves
over time.”
“Based on
business process
modelling to
create business
level keywords
written in natural
language
validated against
content sensitive
validation to
create reusable,
jigsaw pieces
representing the
Solution Under
Test (SUT).”
Test Automation as a Service
* Jonathon Wright, 2010, STARWest Track, ”Test Automation as a Service”, http://www.TaaS.net, 2nd October 2012
@jonathon_wright
#BCS_SIGIST
32
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Introduction to Model Driven Delivery integration (MDDi)
Agile Engineering
Core IT Fluid IT
Agile Engineering
* Adam Mann, Google Innovation Officer, ‘Creating the inter-planetary Internet protocol’, Wired.com, 6th May 2013
Open Innovation
33
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Introduction to Model Driven Delivery (MDDi) - Charter(s)
Test First Delivery (TFD) First Day Delivery (FDDi)
TDD
* Jonathon Wright, 2014, Gartner, http://www.slideshare.net/mobile/Jonathon_Wright/addi-ha-apiytestinghybridcloud
Domain Driven Design (DDD) Domain Driven Delivery (DDDi)
Transcendence Driven Delivery (TDDi)
Test First Delivery (TFD)
34
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Charter – First Day Delivery (FDDi)
Model Driven Delivery (MDDi) integration
*The Advanced Research Projects Agency Network (ARPA) Network, 1969, 2 & 4 Nodes the precursor to the Internet
Business Process Modelling
Blueprint
Business Process Scenarios
Patterns
Business Process Components
RecipesNode Maps Node Learning Node Strengthening
• Model the proposed solution • Learning (Turing machine)
adapting to simulate logic
algorithms
• Validate all possible states
Smart Learning Machines
35
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Charter – Transcendence Driven Delivery (TDDi)
Model Driven Delivery (MDDi) integration
Business Process Scenarios
Patterns Node (View)
• Each Node creation (LOC)
passes requirement, story,
behaviour, acceptance test
• Each Node generates a
testable view (UI, Service,
or Data layer) of the solution
• Models Nodes and Links
Business Process Components
Recipes NodeJS (Controller)
Business Process Modelling
Blueprint Node (Model)
* Visual Studio 2015 CP6 & NodeJS, https://www.visualstudio.com/en-us/news/vs2015-vs.asp http://nodejstools.codeplex.com/
36
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Charter – Domain Driven Delivery (DDDi)
Model Driven Delivery (MDDi) integration
Business Process Modelling
Blueprint
Business Process Scenarios
Patterns
Business Process Components
RecipesNode Discovery Node ValidationNode Systems
• Model the current solution • Validate all possible states• Apply:• Systems Dynamics
(epistemic & systemic
entropy)
• Systems Thinking
• Learn:• Ontology – formal naming,
definition of node types,
properties and
interrelationships
• Domain – universal set of
nodes (domain of discourse)
• Context – nouns/verb
discovery to enable context
sensitive validation.
* Huw Price, 2015, Grid-Tools, http://www.agile-designer.com/resources/model-based-testing-using-agile-designer/
@jonathon_wright
#BCS_SIGIST
37
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Model Based Design (MBD)
Advanced Modelling Simulation Technologies
Watershed management
Pipe-network Management
Prediction of water pollution
Pipe-network analysis
© Hitachi Europe Ltd. 2014. All rights reserved.
Models can be analysed, visualized, and optimized
Models can provide assurance through, analysis, simulation and prediction
* http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi
38
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
• Test Platform as a Service (TPaaS)
– Social Enterprise Collaboration
– Actionable Business Insight
– Enterprise Gamification
– Global Testing Marketplace
– All-Channel Customer Experience• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Test Platform as a Service (TPaaS) Model
39
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Global Testing Marketplace
Instant Scalability, Flexibility and Availability
@jonathon_wright
#BCS_SIGIST *Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011
Test Platform as a Service (TPaaS) Model
G
L
O
B
A
L
M
A
R
K
E
T
P
L
A
C
E
40
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Global Testing Marketplace
Improved Communication, Collaboration and Mobility
*Jonathon Wright, 2012, ‘Test Automation as a Service’ at Fusion, Sydney, 10th September 2012
Test Platform as a Service (TPaaS) Model
Test©
41
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Global Testing Marketplace
Become a Testing Rock Star!
*http://blog.utest.com/2014/11/04/utest-platform-preview-new-dashboard-for-testers-on-paid-projects , 4th Nov 2014
Test Platform as a Service (TPaaS) Model
@jonathon_wright
#BCS_SIGIST
42
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Enterprise Gamification
Service Thinking (Service as a Experience) Business Model
G
A
M
I
F
I
C
AT
I
O
N
E
N
T
E
R
P
R
I
S
E
Shift Left
Shift Up
Shift Down
Shift Right
Test
in
Dev
Business
Acceptance
Testing
Test
Ops
User
Acceptance
Testing
Social
Enterprise
Test Platform as a Service (TPaaS) Model
* http://enterprise-gamification.com/attachments/article/233/Improving%20Software%20Quality%20with%20Gamification%20-%20Full%20Paper.pdf
Business
Enterprise
(CIO, COO, CTO)
43
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Enterprise Collaboration in the Crowd – Real World Example
Test Intelligence 2.0 – Global Testing Insight (UX, Exploratory & Testing in the Wild)
Request1
Run2
Test3
* Real Crowd Tester, Liam Warrilow, 24th Friday 2015 @15:13 (Ramp up to 100 iWatch testers in under 24hrs)
Enterprise CrowdTesting
Testing in the Wild
Microsoft© HQ Seattle
44
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Actionable Business Insight powered by ‘Small Data’
Funnel Virtualisation
BusinessInsight
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
DataData
Data
Data
DataData
* Llyr Jones, 2014, Grid-Tools.com, https://twitter.com/GridTools/status/487264150578790400 , 10th July 2014
Test Platform as a Service (TPaaS) Model
45
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Social Enterprise Collaboration
• Delivering true business value – Not just driving process efficiency
– Across the SDLCi with cross-functional domain tools
– Providing instant actionable business insight
• Supporting individual, team, organizational, and enterprise collaboration
– Integration with the Global Testing Marketplace (GTM) platform
– Produce high-quality reusable business assets in the cloud
– Capture specialist technical and cross-domain knowledge (i.e. DIDO)
– Game mechanics to encourage collaboration (i.e. Enterprise Gamification)
– Produce design patterns that allow business rules to be codified in business-specific meta-languages (i.e. BPMNv2.2, xPDL, XAML)
• Managing software that gets deployed everywhere
– All-Channel Customer Experience
*Jonathon Wright, 2014, ‘Agile Portfolio Management in the Cloud’ presented at Unicom, London, 20th March 2014
Test Platform as a Service (TPaaS) Model
Embracing Agile Portfolio Management in the Cloud
46
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
– H(API)y Testing in the Cloud
– Test First Delivery
– Building bridges in the Cloud
– Cloud 2.0 beyond legacy
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Test Infrastructure as a Service (TIaaS) Model
47
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Building Bridges in the Cloud
“If you build it, they will test” (IYBITWT)
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
48
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Cloud 2.0
Moving beyond the legacy cloud
Test Infrastructure as a Service (TIaaS) Model
select <IaaS>
select <PaaS>
select <SaaS>
select <TaaS>
select <TaaaS>
Cloud Recipe for Solution Under Test
Cloud Recipe (IFTTT)
CloudSystem Automation
YourTestCloud.com
“Heterogeneous test cloud that are open, flexible and extensible”
Testing Cloud
If <Solution Under Test>
Then <Build Cloud 2.0>
Testing Cloud
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
49
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Test First Delivery
Service / Network Virtualisation as part of Test Driven Delivery integration (TDDi)
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
50
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Lean = Open
Open (Digital) Innovation results in Open (Digital) Businesses
Open Digital Platforms
Lean Testing
Enterprise
Practices
BlueprintsSmart
“Sharing intellectual property such as software code or research
rules, adhering to open standards rather than creating your own,
and giving customers the freedom to easily exit your platform”
*Jonathan Rosenberg & Eric Schmidt, 2014, ‘How Google Works’, ISBN: 9781455582341, 23rd September 2014
Open Digital Enterprises
51
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Open Innovation 1.0 - Real World Example
SDLCi - Grid-Tools connected intelligence to PerfectoMobile
*Llyr Wyn Jones, ‘End-to-end vs. interface testing: the mathematics’, 31st March 2015
@jonathon_wright
#BCS_SIGIST
52
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Ha(API)y Testing in the Cloud
• Build Bridges in the Cloud
– Use Cloud Maps to Learn, Discover and Model endpoint(s)
– Create Bridges (VNet to VNet) “If you build it, they will test” (IYBITWT)
– Embrace Test Infrastructure as a Service (TIaaS) in the Hybrid Cloud (TQA™ Lab)
• Avoid Legacy Cloud
– Heterogeneous Test Infrastructure as a Service (TIaaS) that are open, flexible and extensible
– Continuous Build, Integration & Delivery in YourTestCloud
– Embrace Community Test Clouds (Collaborate/Share Recipes)
• First Day Delivery
– Enable Day-zero Performance, Penetration & Security Testing as a Service
• Support for powerful Test Automation as a Service (TAaaS)
– UI Testing that is Scalable and Cost-Efficient
– API Testing that is Repeatable, Reliable and Fast
Unlock true delivery innovation
*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014
Test Infrastructure as a Service (TIaaS) Model
53
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
– Test Automation in the Cloud
– AMMi© Assessment Model
– First Day Automation
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Test Automation as a Service (TAaaS) Model
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
54
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
First Day Automation: Test Automation Experiences Book
* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences/, ISBN-10: 0321754069, 9th January 2012
Test Automation as a Service (TAaaS) Model
55
@jonathon_wright
#BCS_SIGIST www.sigist.org.ukhttp://partners.telerik.com/docs/default-source/Case-Studies/telerik-test-automation-in-the-cloud_automation-development-servicesBB95938E9FB5.pdf
Test Automation as a Service (TAaaS) Model
Test Automation in the Cloud: Case Study
56
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
AMMi© Assessment Model
The digital journey toward technical delivery enlightenment
AMMi Phase Target Criteria
Level 0 Accelerating Automated Test Lifecycle Methodology (ATLM)
Level 1 Traditional Framework Driven (Gen 1-5)
Level 2 Managed Processes are planned, performed, measured, and controlled
Level 3 Sustaining Automation process defined and validated against international
standards (ISO-29119)
Level 4 Quantified Predictability of automation process performance
Level 5 Optimising Automation process variation and statistical predictability
* Source: AMMi®(http://www.AMMi.org.uk) + Test Automation Patterns (http://TestAutomationPatterns.wikispaces.com)
Test Automation as a Service (TAaaS) Model
57
@jonathon_wright
#BCS_SIGIST www.sigist.org.ukFunctional Testing as a Service (FTaaS) Model
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service
– Shift everywhere (UP, DOWN, LEFT, RIGHT)
– Continuous, Integration, Build & Delivery
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
58
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Shift Everywhere
Test in Business, Test in Dev, Test Ops, Test in the Wild
Functional Testing as a Service (FTaaS) Model
Test in Business Test in Dev Test in the Wild
59
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Functional Testing in UFT 12 and beyond (i.e. LeanFT)
* https://www.packtpub.com/application-development/advanced-qtp-115-test-engineers-cookbook
@jonathon_wright
#BCS_SIGISTFunctional Testing as a Service (FTaaS) Model
60
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service
– Performance Testing in the Cloud
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Performance Testing as a Service (PTaaS) Model
61
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Performance Testing in the Cloud
PTaaS – Microsoft – Visual Studio Team Foundation Service
Performance Testing as a Service (PTaaS) Model
62
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Performance Testing in the Cloud
@jonathon_wright#BCS_SIGIST Performance Testing as a Service (PTaaS) Model
PTaaS – HP - StormRunner (TrueAPI/TrueClient)
63
@jonathon_wright
#BCS_SIGIST www.sigist.org.ukMobile Testing as a Service (MTaaS) Model
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service
– Mobile Testing in the Cloud
– Continuous, Mobile, Build & Delivery
– High Volume Mobile (HVMAT)
• Security Testing as a Service (STaaS)
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
64
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Mobile Testing in the Cloud
@jonathon_wright#BCS_SIGIST
Real Mobile Devices (PerfectoMobile) Test Studio for iOS (Telerik)
* Telerik.com, http://www.telerik.com/mobile-testing, 4th November 2014
Mobile Testing as a Service (MTaaS) Model
65
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Continuous, Mobile, Build & Delivery
MobileCloud™
@jonathon_wright#BCS_SIGIST
* PerfectoMobile.com, http://www.perfectomobile.com/resources/papers/shift-app-testing-left-paper, 4th November 2014
Performance Testing as a Service (PTaaS) Model
66
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
High Volume Mobile Automated Testing (HVMAT)
On demand in the cloud featuring Network Virtualisation
*Jonathon Wright, 2013, ‘Test Automation in the Cloud’ for EuroSTAR, Gothenburg, Sweden, 23rd July 2013
Mobile Testing as a Service (MTaaS) Model
67
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
• Test Platform as a Service (TPaaS)
• Test Infrastructure as a Service (TIaaS)
• Test Automation as a Service (TAaaS)
• Functional Testing as a Service (FTaaS)
• Performance Testing as a Service (PTaaS)
• Mobile Testing as a Service (MTaaS)
• Security Testing as a Service
– Security Testing in the Cloud
Business Delivery Management - ‘Testing as a Service’ layer
Testing as a Service - Models
Security Testing as a Service (STaaS) Model
68
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Security Testing in the Cloud
How to prevent epistemic/systemic failure in the cloud from geo-realistic DoS attacks? Application
Security
@jonathon_wright
#BCS_SIGIST
Infrastructure
Testing
Secure Coding
Mobile Security
Database Security
Information
Security
Our services identify
configuration and
deployment flaws which
may be introduced
when integrating mobile
solutions into an
enterprise environment.
We also perform
manual security testing
and analysis of smart
phone and tablet based
apps. Typically this will
encompass both the
client application and
server side services
supporting the app.
*John Denneny, MD, http://www.pentest.co.uk/whitepapers.html , 5th November 2014
Security Testing as a Service (STaaS) Model
Mobile Security
69
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Testing as a Service – Models - Summary
BusinessAssurance
BusinessQuality
BusinessTesting
Business Assurance provides the visibility and
transparency throughout the solution delivery lifecycle• Digital Enterprise
• Business on a Page
• Business Delivery Management
• Business Intelligence (Actionable Business Insight)
• Business Innovation (Enterprise Gamification)
Business Quality allows enforceable quality across the
entire solution delivery lifecycle owned by everyone• Business Lifecycle Management
• Business Solution Discovery / Delivery
• Business Enterprise Collaboration
• Business Acceptance Testing
Business Testing
• Testing as a Service - Models
• Solution Delivery Lifecycle integration (SLDCi)
• Test First Delivery / First Day Testing
• Continious, Integration, Build & Delivery
• Shift Everywhere (UP/DOWN/LEFT/RIGHT)
• All Channel Customer Experience
70
@jonathon_wright
#BCS_SIGIST www.sigist.org.uk
Hitachi ConsultingHitachi’s Global Management Consulting & IT Services Business.
$740
MillionFY13 Revenue
6,500experienced &
dedicated
professionals
Better Approach. Better Results. Better Experience.
$10 Billion
results
© Hitachi Europe Ltd. 2014. All rights reserved.
top related