achieve competitive advantage with ibm devops · rational functional tester •record, enhance, and...

42
Rational Software Rational Software Rational Software Rational Software Achieve Competitive Advantage with IBM DevOps Quality is not Accidental IBM Competitive Project Office © 2014 IBM Corporation

Upload: others

Post on 05-Apr-2020

20 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Achieve Competitive Advantage with IBM DevOps

Quality is not Accidental

IBM Competitive Project Office© 2014 IBM Corporation

Page 2: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Introduction

Break Down the Barriers with Improved Planning and Collaboration

Define, Manage, and Track your Requirements and Designs

Agenda

© 2014 IBM CorporationIBM Competitive Project Office 2

Integrated Change, Build and Deploy is What Sets Us Apart

Quality is not Accidental

Bridge the Gap between Development and Operations

Put your Business in Motion - Become a Mobile Enterprise

Page 3: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Errors caught early cost less to fix . . .

“30%30%30%30% of project costs are due to rework and poor execution of requirements”

“50%50%50%50% of outsourced projects are expected to under perform”

“34%34%34%34% of software projects are deemed successful costing $300B annually”

• Bugs/failures are exponentially more expensive the longer they go uncaught

• Extends to analysis, requirements, development, and everywhere across the application lifecycle

© 2014 IBM CorporationIBM Competitive Project Office 3

Rela

tive

Co

st t

o R

ep

air

AcceptanceUnit TestCoding Maintenance

1.5x

Stage in which Error is Discovered

and poor execution of requirements” to under perform”

Analysis Design

5x10x

20x

50x

200x

Page 4: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Quality management must span the lifecycle

Release to ProductionTesting

Acceptance

System

Performance

Analysis

Requirements

Main

ten

an

ce

© 2014 IBM CorporationIBM Competitive Project Office 4

Integration Testing

Unit Testing

Functional Testing

Coding

Main

ten

an

ce

All of these tests must be planned. All of these tests must be planned. All of these tests must be planned. All of these tests must be planned. Artifacts Artifacts Artifacts Artifacts created by tests need to be managed, created by tests need to be managed, created by tests need to be managed, created by tests need to be managed, linked to other development artifacts, and linked to other development artifacts, and linked to other development artifacts, and linked to other development artifacts, and analyzed.analyzed.analyzed.analyzed.

Page 5: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Definition of common test types

• Unit testing – Test individual units of source code

– Determines if code works as intended (fit for use)

• Functional testing– Ignores the internal mechanism of a system or component

– Focuses solely on the outputs generated in response to selected inputs and execution conditions

– Can be manual or automated

© 2014 IBM CorporationIBM Competitive Project Office 5

– Can be manual or automated

• Performance testing– Evaluates the compliance of a system or component with specified

performance requirements

• Integration testing– Input: modules that have been unit tested

– Groups modules in larger aggregates and applies tests defined in an integration test plan

– Output: integrated system ready for system testing

Page 6: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager provides a management hub for quality artifacts

RQM Client: Web browser

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defectsReporting & Dashboard:

Status & progress tracking,

Customizable live

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

© 2014 IBM CorporationIBM Competitive Project Office 6

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Page 7: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager – test planningThe test plan is the core artifact that

contains all information pertaining to

the quality management effort

RQM Client: Web browser

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defectsReporting & Dashboard:

Status & progress tracking,

Customizable live

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Team Collaboration:

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Storyboards

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

Work Items

© 2014 IBM CorporationIBM Competitive Project Office 7

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Page 8: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Central location for business objectives, requirements, resources, platform and exit criteria to name a few

• Defined Responsibilities

Individual sections

• Goal-Oriented

Formalized and

Rational Quality Manager’s test plan

© 2014 IBM CorporationIBM Competitive Project Office 8

Formalized and documented exit criteria

• Extensible

Add sections, import custom data

• Snapshot version control to track plan history

Page 9: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Planning fields for test artifacts to help prioritize, schedule and manage testing activities

Test prioritization helpers

• Test Suite/Case Execution Record section in test plans

Traceability and execution progress

views

Comprehensive test plan

Unify the team through real-time collaboration and clear objectives

© 2014 IBM CorporationIBM Competitive Project Office 9

Page 10: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager – test constructionRQM has built-in function to allow teams

to track lab assets and configurations, provision lab assets, and schedule and reserve the assets

RQM Client: Web browser

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defectsReporting & Dashboard:

Status & progress tracking,

Customizable live

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Team Collaboration:

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Storyboards

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

Work Items

© 2014 IBM CorporationIBM Competitive Project Office 10

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Page 11: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Requirements management and integration

Requirements-driven testing with RQM

New and New and New and New and UpdatedUpdatedUpdatedUpdated

RequirementsRequirementsRequirementsRequirements

Requirement Requirement Requirement Requirement update notificationupdate notificationupdate notificationupdate notification

Duplicate Duplicate Duplicate Duplicate RequirementRequirementRequirementRequirementPreventionPreventionPreventionPrevention

Requirement Requirement Requirement Requirement impact analysisimpact analysisimpact analysisimpact analysis

Dynamic Dynamic Dynamic Dynamic Test PlanTest PlanTest PlanTest Plan

© 2014 IBM CorporationIBM Competitive Project Office 11

Requirements management and integration

• Advanced requirement analysis prevents duplicates and duplicate efforts

• Review and approvals of tasks across projects

• Asset versioning with dynamic update

• Define requirements in rich text

Review & Review & Review & Review & ApproveApproveApproveApprove

Page 12: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Create test scripts manually or automatically• Record user interactions with an application under test using RFT• Recording is captured as a RQM manual test script in English natural

language with screenshots, independent from RFT• Resulting manual test scripts are executed manually using RQM only• Supports Web and Java applications running on Windows

Author manual test scripts step-by-step or record them with IBM Rational Functional Tester

© 2014 IBM CorporationIBM Competitive Project Office 12

Accelerate and streamline manual

testing creation

Page 13: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• RQM includes test lab management capability

– Capture and organize test lab inventory

– Manage requests and reservations of test lab components

– Deploy test tools to lab machines

Analyze usage patterns to optimize lab utilization

Built-in test lab management

© 2014 IBM CorporationIBM Competitive Project Office 13

– Analyze usage patterns to optimize lab utilization

– Do comprehensive reporting on test assets

• Integrates with virtual automation platforms, e.g., Surgient and VMLogix

• RQM can integrate with 3rd party test lab management products

Page 14: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Manage

– Verify that JKE has the resources required to fulfill their test plan

• Deploy

– Deliver the configurations JKE’s

JKE gains more test time

40% Lab Setup40% Lab Setup 60% Test Time60% Test TimeBefore lab automation

Non-automated tasks or steps

© 2014 IBM CorporationIBM Competitive Project Office 14

configurations JKE’s teams require for test

• Optimize

– Analyze patterns to minimize cost and maximize utilization

Work smarter, save on test lab overhead, infrastructure and duration costs

60% Test Time60% Test Time

automation

With lab automation

Application setup with RQM (50% reduction)

20% Setup20%

Setup20% Free20% Free

Page 15: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager – test executionRQM provides RQM provides RQM provides RQM provides builtbuiltbuiltbuilt----in in in in manual manual manual manual test authoring, test authoring, test authoring, test authoring, , execution, and management along with , execution, and management along with , execution, and management along with , execution, and management along with automated testing from various sourcesautomated testing from various sourcesautomated testing from various sourcesautomated testing from various sources

RQM Client: Web browser

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defectsReporting & Dashboard:

Status & progress tracking,

Customizable live

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Team Collaboration:

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Storyboards

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

Work Items

© 2014 IBM CorporationIBM Competitive Project Office 15

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Page 16: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

JKE’s manual testers can’t keep pace with the rapid releases of JKE’s agile development process

They need a tool to provide assistance to the manual tasks involved with manual testing (i.e. data entry, data validation, reuse)

JKE can’t keep up with their manual tests –they need tool automation, reuse, and management

Manual testing Manual testing Manual testing Manual testing

© 2014 IBM CorporationIBM Competitive Project Office 16

Manual testing Manual testing Manual testing Manual testing can’t keep pacecan’t keep pacecan’t keep pacecan’t keep pace!!!!

They must They must They must They must automate their automate their automate their automate their manual testing manual testing manual testing manual testing process!process!process!process!

TimeTimeTimeTime

Tests

Tests

Tests

Tests

Build 1Build 1Build 1Build 1

...take ...take ...take ...take more timemore timemore timemore time

...the new ...the new ...the new ...the new

tests...tests...tests...tests...

Build 2Build 2Build 2Build 2 Build 3, 4, 5 …Build 3, 4, 5 …Build 3, 4, 5 …Build 3, 4, 5 …

Rerun the first Rerun the first Rerun the first Rerun the first tests and...tests and...tests and...tests and...

Page 17: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

RQM’s manual tester assists manual test case executionRich Text Editor

Step by step capture and execution

Reusable keywords

© 2014 IBM CorporationIBM Competitive Project Office 17

Assisted data entry and validation

Automatic defect creation – send directly to RTC (or your own CCM tool)

Page 18: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Use a mobile app for off-line test execution• Beneficial for testing actual products or systems while being in the field or in an

isolated test lab without connectivity

• Mobile application allows tests to be run while disconnected capturing verdicts and results

© 2014 IBM CorporationIBM Competitive Project Office 18

Connect app to server and download test scripts

Perform script steps and enter results (pass/fail, comments, pictures)

Complete steps and upload test record to server when device has connectivity

Page 19: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational also offers a complete suite of testing tools

10101010

Rational Functional Tester

Rational Functional Tester

• Record, enhance, and execute automated tests

• Java and/or Visual Studio generated test scripts

• Record, enhance, and execute automated tests

• Java and/or Visual Studio generated test scripts

Rational Integration Tester

Rational Integration Tester

• Web service visual editors• WS-BPEL testing• Code-free testing

• Web service visual editors• WS-BPEL testing• Code-free testing

Rational Performance Tester

Rational Performance Tester

• Root cause analysis• Performance loads• Automated script capture

and playback

• Root cause analysis• Performance loads• Automated script capture

and playback

Adapters

Rational Test WorkbenchRational Test WorkbenchRational Test WorkbenchRational Test Workbench

© 2014 IBM CorporationIBM Competitive Project Office 19

Jazz Team Server

Test Planning:Test Planning:Test Planning:Test Planning:Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope,

Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment

Test Planning:Test Planning:Test Planning:Test Planning:Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope,

Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment

Test Execution:Test Execution:Test Execution:Test Execution:Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record

test results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defects

Test Execution:Test Execution:Test Execution:Test Execution:Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record

test results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defects Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, RealRealRealReal----time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality

auditauditauditaudit

Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, RealRealRealReal----time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality

auditauditauditaudit

Test Construction:Test Construction:Test Construction:Test Construction:RequirementRequirementRequirementRequirement----driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage,

Manual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab management

Test Construction:Test Construction:Test Construction:Test Construction:RequirementRequirementRequirementRequirement----driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage,

Manual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab management

Team Collaboration:Team Collaboration:Team Collaboration:Team Collaboration:Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task management, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member on----boardingboardingboardingboarding

Team Collaboration:Team Collaboration:Team Collaboration:Team Collaboration:Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task management, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member on----boardingboardingboardingboarding

Rational Quality ManagerRational Quality ManagerRational Quality ManagerRational Quality Manager

Page 20: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally managed Automated Functional, Performance and managed Automated Functional, Performance and managed Automated Functional, Performance and managed Automated Functional, Performance and Integration/Virtualization TestingIntegration/Virtualization TestingIntegration/Virtualization TestingIntegration/Virtualization Testing

• Rational Quality Manager provides complete lifecycle integration

– Automatically populate defects, scripts, results, etc.

– Testers don’t need to learn how to use RQM – their artifacts are reused

RQM integration with Rational testing tools extends quality management

© 2014 IBM CorporationIBM Competitive Project Office 20

– Testers don’t need to learn how to use RQM – their artifacts are reused from RQM that connects via adapters to the testing tools

– All artifacts can be linked to RDNG and RTC from RQM for greater traceability

• Build and execute test cases with scripts from any combination of testing tools

– Rational Functional Tester, Rational Performance Tester, Rational Integration Tester, and Rational Test Virtualization Server

– Example: RFT or RIT scripts combined in a performance test case

– Example: RTVS stub deployed and started from RQM

Page 21: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Definition: Definition: Definition: Definition: Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions.1

• Automated test scripts are more productive

• Use to test new code or for regression testing on changed code

Automated functional testing

© 2014 IBM CorporationIBM Competitive Project Office 21

1[IEEE 90] Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE

Standard Computer Glossaries. New York, NY: 1990.

Record

Test ScriptEnhance

Recorded Script

Execute

Test Scripts

Page 22: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

• Use Rational Functional Tester

Record/Enhance/Execute scripts on Windows/Linux

Functional test any .NET, Web, Java application, Siebel, SAP, and many more

Rational Functional Tester

.NET, Browser, Java UI

© 2014 IBM CorporationIBM Competitive Project Office 22

• Use Rational Functional Tester Extension for Terminal-based Applications

Record/Enhance/Execute scripts on Windows

Functional test System z terminal based applications

System z Terminal UI

Page 23: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Definition: Testing conducted to evaluate the compliance of a system or component with specified performance requirements1

The evaluation of a system’s capability to provide acceptable response times and function while serving a large number of clients

Also known as Load Testing, Capacity Testing, or Stress Testing

Performance testing

16 hits/sec

Page hit rate

© 2014 IBM CorporationIBM Competitive Project Office 23

1[IEEE 90] Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY: 1990.

12 hits/sec

Page hit rate

Time

Increase user load

Died Died Died Died with with with with 700 700 700 700 usersusersusersusers

700

16 hits/sec

Time

Increase user load2,000

Well Well Well Well behaved behaved behaved behaved up to up to up to up to 2,000 users2,000 users2,000 users2,000 users

Page 24: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Use Rational Performance Tester to:

• Develop and execute scripts on Windows or Linux

• Collect server resource data and application performance data in the same product

Rational Performance Tester

Any Web Application

© 2014 IBM CorporationIBM Competitive Project Office 24

performance data in the same product

• No programming knowledge necessary to create/execute test

Perform tests on a variety of applications

Root Cause Analysis – identify both source code and application tier causing the bottleneck

Page 25: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

DEMO 1: Integration between RQM and RFT

He plays the script back and watches the behavior of the

Tanuj creates an automated functional test with Rational Functional Tester recorder

© 2014 IBM CorporationIBM Competitive Project Office 25

He analyses the test results to check for any errors or defects

He plays the script back and watches the behavior of the application

Page 26: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

We’ve encountered We’ve encountered We’ve encountered We’ve encountered some challenges some challenges some challenges some challenges as our IT environment as our IT environment as our IT environment as our IT environment has become has become has become has become more more more more complex and the development complex and the development complex and the development complex and the development teams deliver teams deliver teams deliver teams deliver more changes more changes more changes more changes faster. faster. faster. faster. Do you have a solution that could Do you have a solution that could Do you have a solution that could Do you have a solution that could help us accelerate integration help us accelerate integration help us accelerate integration help us accelerate integration testing?testing?testing?testing?

Yes, we do Yes, we do Yes, we do Yes, we do –––– Rational Test Rational Test Rational Test Rational Test Workbench and Servers (Green Workbench and Servers (Green Workbench and Servers (Green Workbench and Servers (Green Hat) help significantly reduce Hat) help significantly reduce Hat) help significantly reduce Hat) help significantly reduce testing time and improve quality testing time and improve quality testing time and improve quality testing time and improve quality with its innovative test with its innovative test with its innovative test with its innovative test virtualization.virtualization.virtualization.virtualization.

© 2014 IBM CorporationIBM Competitive Project Office 26

IBMQA Manager

Page 27: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Test Workbench (RTW) and ServersAddress three major challenges in quality managementAddress three major challenges in quality managementAddress three major challenges in quality managementAddress three major challenges in quality management

Development Agility Development Agility Development Agility Development Agility and Speedand Speedand Speedand Speed

Increasing Increasing Increasing Increasing Cost of QualityCost of QualityCost of QualityCost of Quality

Increasing Increasing Increasing Increasing Development ComplexityDevelopment ComplexityDevelopment ComplexityDevelopment Complexity

Public CloudPublic CloudPublic CloudPublic CloudPrivate CloudPrivate CloudPrivate CloudPrivate Cloud

CollaborationCollaborationCollaborationCollaboration

Web/Web/Web/Web/InternetInternetInternetInternet

Routing Routing Routing Routing ServiceServiceServiceService ContentContentContentContent

ProvidersProvidersProvidersProvidersEJBEJBEJBEJB

SharedSharedSharedShared

Business Business Business Business PartnersPartnersPartnersPartners

TraditionalTraditionalTraditionalTraditionalTesting Can’t Keep Testing Can’t Keep Testing Can’t Keep Testing Can’t Keep

up with Agile up with Agile up with Agile up with Agile VelocityVelocityVelocityVelocity

© 2014 IBM CorporationIBM Competitive Project Office 27

* Source:http://www.sei.cmu.edu/about/message/

Heterogeneous EnvironmentsHeterogeneous EnvironmentsHeterogeneous EnvironmentsHeterogeneous Environments

Data WarehouseData WarehouseData WarehouseData Warehouse MainframeMainframeMainframeMainframeEnterprise Enterprise Enterprise Enterprise Service BusService BusService BusService Bus

Directory Directory Directory Directory IdentityIdentityIdentityIdentity

File File File File systemssystemssystemssystems

ThirdThirdThirdThird----partypartypartypartyServicesServicesServicesServices PortalsPortalsPortalsPortals

SharedSharedSharedSharedServicesServicesServicesServices

ArchivesArchivesArchivesArchives

Messaging Messaging Messaging Messaging ServicesServicesServicesServices

Other costsOther costsOther costsOther costs

Testing costsTesting costsTesting costsTesting costs

Page 28: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Traditional process involves the “big bang” integration testing after unit tests

System Tests

Requirements

Unit TestsBig Bang

© 2014 IBM CorporationIBM Competitive Project Office 28

UAT

OAT

Performance

Legend: quality gate

• Multiple defects introduced at once

• More unknowns – greater risk

• Result: more re-work, more expensive

Page 29: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

With Rational Test Virtualization Server and Rational Integration Testing, “big bang” is eliminated from the process

Requirements

Unit TestsBig Bang

Integration Tests

System Tests

System

Tests

• Accelerates testing

• Reduces costs

• Lowers risk

© 2014 IBM CorporationIBM Competitive Project Office 29

Performance

UAT

OAT

Tests

Page 30: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Test Virtualization Server enables incremental integration testing

Actual Component

Stubbed Component

• Units are introduced into the continuous integration cycle in a prioritized, controlled fashion.

• Units not yet built can be simulated and tested against.

© 2014 IBM CorporationIBM Competitive Project Office 30

Incremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration Testing

Page 31: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Test Workbench

• Combination of

Rational Test Workbench

Eclipse and Mobile clients

Rational Functional Tester

Rational Performance Tester

Rational Integration Tester

Rational Test Virtualization Server

• Enables sharing and reuse of test environments across the enterprise

− Sharing and reuse of virtualized components/services

− Test environments available on demand

− Enables automated build test with

Rational Test Workbench and Rational Test Virtualization Server

© 2014 IBM CorporationIBM Competitive Project Office 31

Rational Integration Tester

Rational Test Control Panel

• Virtualizes services - no coding required!

• Reuse existing RFT and RPT assets

• Integrated with RDNG, RQM, and RTC for ALM traceability

− Enables automated build test with enterprise integration testing scenarios

• Reduce test infrastructure cost and dependencies

• Leverage physical, virtual, or cloud test environments for different types of testing workloads

Page 32: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

IBM Rational Quality Management with Rational Integration Tester and Rational Test Virtualization Server The leading Agile Quality Management Solution

ManageCreate Build ReportExecute

IBM Collaborative Application Lifecycle Management

Test Management

Rational Quality Manager

Quality Dashboard

RequirementsManagement Defect

Management

© 2014 IBM CorporationIBM Competitive Project Office 32

TestVirtualization

Storage

Collaboration

Search & QueryDiscovery

Administration: Users, projects, process

Presentation:

Mashups

Best Practice Processes

Manage

Test Lab

Create

Plan

Build

Tests

Report

Results

Execute

Tests

Open Lifecycle Service Integrations

PerformanceTesting

PerformanceTesting

FunctionalTesting

UnitTesting Security and

ComplianceIntegrationTesting

Page 33: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

DEMO 2: Integration tests and test virtualization

He starts the stub in RIT and tests the access to it by

Tanuj uses Rational Integration Tester to create a stub to simulate access to a Web service

© 2014 IBM CorporationIBM Competitive Project Office 33

He deploys the stub to Rational Test Virtualization Server so it can be accessed by other users

He starts the stub in RIT and tests the access to it by interacting with the application under test

Page 34: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager reportingDynamic reports based on Jazz Common Reports show how quality management artifacts are linked throughout the lifecycle

RQM Client: Web browser

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment

Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defectsReporting & Dashboard:

Status & progress tracking,

Customizable live

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Storyboards

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

Work Items

© 2014 IBM CorporationIBM Competitive Project Office 34

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Page 35: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager reporting

Common reporting across RQM, RDNG and RTC means all QM artifacts are transparent throughout the entire lifecycle

© 2014 IBM CorporationIBM Competitive Project Office 35

Page 36: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational Quality Manager – team collaborationTeam collaboration features allow QM team to easily follow best practices and make everyone more productive

RQM Client: Web browser

Rational DOORS Next Generation

Rational DOORS Next Generation

Collections

Requirements

Storyboards

Rational Software

Architect Design Manager

Rational Software

Architect Design Manager

Designs

Models

Rational Team Concert

Rational Team Concert

Release Plans

Iterations

Work Items

Test Planning:

Comprehensive test plan,

Shared objectives, Scope,

Timeline, Resources, Risk

assessment

Test Execution:

Manual test execution, Use

test automation tools, Record

test results, Submit & track

defects

Test Construction:

Requirement-driven testing,

Test environments

coverage, Manual test

authoring, Test lab

management

Reporting & Dashboard:

Status & progress tracking,

Customizable live

© 2014 IBM CorporationIBM Competitive Project Office 36

Jazz Team Server

RTCRDNG RQM

Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,

Administration, Reporting, etc…

RDNG REST

Services: Rqmts

Management

RTC REST

Services: Change

Management

RQM REST

Services: Quality

Management

RSA DM

Storyboards

Use cases

Diagrams

Work Items

Source Code

RSA DM REST

Services: Design

Management

Team Collaboration:

Process enactment and

enforcement, reviews, task

management, Rapid team

member on-boarding

Customizable live

dashboard, Real-time

metrics and reports,

Compliance and quality

audit

Page 37: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Rational CLM quality management adoption model

Tactical, opportunistic Strategic, systematic

Cost ReductionCost ReductionCost ReductionCost Reduction� Increase

Automation� Reuse Test assets� Optimize

infrastructure

Broaden ScopeBroaden ScopeBroaden ScopeBroaden Scope� Drive QM to meet

business needs� Extend QM to the

overall ALM� Consolidate QM

information

StandardizationStandardizationStandardizationStandardization� Define and track

organizational quality metrics

� Identify and promote Best Practices

Realizing StrategyRealizing StrategyRealizing StrategyRealizing Strategy� Drive Business

decisions leveraging product quality

� Use Quality as competitive differentiator

© 2014 IBM CorporationIBM Competitive Project Office 37

Cost focusCost focus Value focusValue focus

information� Facilitate

Collaboration with other stakeholders

� Manage Test Data

Practices� Promote

repeatability

differentiator� Drive continuous

quality improvements and real time decisions.

RQMRQMRQMRQM RQM + RTC + RQM + RTC + RQM + RTC + RQM + RTC + RDNGRDNGRDNGRDNG

Organizations often move through a set of phases as they adopt Quality Management

Page 38: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Sizing up the competition

As always, I want to know what the As always, I want to know what the As always, I want to know what the As always, I want to know what the competition is up to. We are already competition is up to. We are already competition is up to. We are already competition is up to. We are already using HP Quality Center. Why should using HP Quality Center. Why should using HP Quality Center. Why should using HP Quality Center. Why should JKE choose IBM over HP Quality JKE choose IBM over HP Quality JKE choose IBM over HP Quality JKE choose IBM over HP Quality Center?Center?Center?Center?

Let me show you why IBM is Let me show you why IBM is Let me show you why IBM is Let me show you why IBM is the leader in Quality the leader in Quality the leader in Quality the leader in Quality Management spanning the Management spanning the Management spanning the Management spanning the entire application lifecycle and entire application lifecycle and entire application lifecycle and entire application lifecycle and key questions to ask the key questions to ask the key questions to ask the key questions to ask the competition.competition.competition.competition.

© 2014 IBM CorporationIBM Competitive Project Office 38

QA Manager

Page 39: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

What you should ask HP

Does HP ALM have a built-in test lab manager?

Can HP manage physical and virtual assets from within HP ALM?

© 2014 IBM CorporationIBM Competitive Project Office 39

How big is HP’s partner network for integration (i.e. how many different products can be integrated)?

Can functional tests be automated across physical and virtual machines easily?

How complete is reporting across all artifacts in the application lifecycle (i.e. can I create reports that span development artifacts, quality artifacts, and requirements?)

Page 40: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

How complete is testing in Atlassian’s solution?

• What kinds of tests can Atlassian run beyond simple JUnit?

• Does Jira Capture (Atlassian plugin, formerly called Bonfire) support test plans or test cases without requiring additional add-ons?

What you should ask Atlassian

© 2014 IBM CorporationIBM Competitive Project Office 40

ons?

• What functional and performance tests does Atlassian support?

• Are there any test lab management capabilities with Atlassian?

What types of defect creation can be automated?

• Can Atlassian’s solution create defects directly from the build engine?

• Can Atlassian automatically populate data if I create a defect in-context using other tools?

Page 41: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

Complete coverage of all quality management artifacts throughout the entire ALM

Seamless integrations with testing tools

• IBM Rational/Tivoli products

Summary

© 2014 IBM CorporationIBM Competitive Project Office 41

• IBM Rational/Tivoli products

• Rational Test Workbench and Servers (formerly Green Hat)

• 3rd party integrations via OSLC and REST

A complete Quality Management environment

• Detailed test planning

• Requirements driven testing is built-in

• Built-in manual tester

• Built-in test lab manager

• Comprehensive reporting capabilities

Page 42: Achieve Competitive Advantage with IBM DevOps · Rational Functional Tester •Record, enhance, and execute automated tests •Java and/or Visual Studio generated test scripts Rational

Rational SoftwareRational SoftwareRational SoftwareRational Software

© 2014 IBM CorporationIBM Competitive Project Office 42

© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

www.ibm.com/software/rational