rqm best practices

8/17/2019 RQM Best Practices

1/52

© 2015 IBM Corporation

IBM Rational QualityManager:

Tips, Tricks and BestPractices

John Nason – RQM Chief Programmer

Sachin Patel – RQM Dev Lead


2/52

22

IBM Rational Quality Manager (RQM)Tips, Tricks and Best Practices

Intro To RQM

Sticking Points

Best Practices: SetupBest Practices: Planning

Best Practices: Execution

Advanced Topics: DevOps

Advanced Topics:Command Line Test

Adapter


3/52

Intro to RQM


4/52

4

Rational Quality Manager: Central Hub for QualityManagement

Rational Quality Manager

Test Planning

Test Construction

Test Execution

Team Collaboration

Reporting & Dashboard

RequirementsManagement

DefectManagement

Rational DNG

Rational DOORSRational RequisitePro

Rational Team Concert

Rational ClearQuest

FunctionalTesting Performance

Testing Integration TestingandTest Virtualization

TestData

Security andCompliance

Rational Functional Tester

Worksoft Certify RationalPerformanceTester

RationalTest Workbench

AppScan

Optim


5/52

5

Rational Quality Manager: A Closer Look

Test Planning

Comprehensive test plan

Shared objectives

Scope, Timeline, Resources

Risk assessment

Test Construction Requirement driven testing

Test environments coverage

Manual test authoring

Test lab management

Test Execution

Manual test execution

Use test automation tools

Record test results

Submit & track defects

Reporting & Dashboard Status and progress tracking

Customizable live dashboard

Real-time metrics and reports

Compliance and quality audit

Team Collaboration Process enactment and enforcement

Review and approval

Task management

Rapid team member on-boarding


6/52

Sticking Points


7/52

Sticking Point: Execution Records

• New users, especially those coming from other test

management tools, are often confused about the Test ExecutionRecord (TCERs & TSERs)

• Execution Records associate the following with a test:

• Who (owner)

• What (test case or suite)

• When (iteration/milestone)

• Where (Plan & Environment)

• Testers “run” TCERs or TSERs, not test cases or suites

7


8/52


• Reporting generally is against the “last result” of a given TCER

• Progress tracking views (more later) work against TCERs

• Tip: from the ER view of a test plan use “Duplicate this ExecutionRecord to (this test plan | another test plan)” to quickly planregression testing, rerun failed tests, or those that you did not get to

8


9/52


• A Test Suite Execution Record tracks running a test suite

• Test Suites are generally used for:

− Regression buckets

− Test scenarios with interdependencies

• Test Suite Execution Records are composed of Test CaseExecution Records

− Note: running a test standalone with the same iteration,plan, test case, and environment “counts” the same as a

test that's part of a suite with the same characteristic Last result is what's used in reporting

• Tip: plan your suites at the same time as your cases for accurateprogress tracking!

9


10/52

Best Practices:Setup


11/52

11

Best Practices: SetupCustomizing the Menu

You can customize the menu to only show items relevant to yourorganization. Find this in Admin > Manage This Project Area > MenuOrganization.


12/52

12

Best Practices: SetupCustomizing the MenuMenu after preceding customization

Menu choices reduced

Drop down choices reduced


13/52

13

Best Practices: Setup Artifact Templates Customize sections present Add new sections Add default content to sections, e.g. “How To”, best practices, etc. Tip: Replace your wiki with self-documenting process in the tool!


14/52

14

Best Practices: SetupCustomizing Permissions

Permissions can be viewed andconfigured by Role or Operation.


15/52

15

• Each artifact type comes with a predefined set of state transitions,which can be customized

• Each artifact supports a set of pre-defined state groups, which canbe used to write process enforcement against

• Each state will then map to each state group

• Transition between states can be customized too

• Can assign process advisers on each state that act aspre-conditions or follow-up actions

Draft

Readyfor Review

Approved

Retired

Readyfor review

Approve

Retire

Reject

Returnto Approve

Reopen

Tailor integrated process to fit your needs

Best Practices: SetupCustom Workflows


16/52

16

Best Practices: SetupProcess Advisors

- Process Advisors provide fine grained control over product behavior andenforce organizational best practices.- Require approvals before state transitions- Require tests to be in a specific state before running them- Configure (extensible) e-Signatures- Automatically lock and unlock test artifacts


17/52

17

• Ability to extract a process template from an existing QM project area

• A QM process template includes:

– Roles and permissions

– Test artifacts workflow customization

– Test artifacts templates

– QM project properties and preferences such as custom attributes, categoriesor quality objectives

• Process template can be reused to quickly create new QM project area

Reuse best practices and quickly setup new projects

Best Practices: SetupProcess Templates


18/52

Best Practices:Planning & Construction


19/52

1919

Test Planning and ConstructionTest Case and Test Suite Categories

RQM now provides functionality to allow a more

comprehensive parent child relationship between categories

Sub categoriescan also beassociated tomultiple parentsfor re-use


20/52

2020


Pre-defined filter queries based on test case categories.


21/52

2121

Saved queries canbe promoted to themenu



22/52

2222


Test Case and Suite categories are visible in Execution Record tables

Saved execution recordqueries can be promotedto the dashboard.

Product quality and defectslisted per test case wherefound.

Group by capabilityavailable. Filtering byProduct and/or capability


23/52

2323

Test Casecategoryfiltersavailable


Detailedpointprogressreport


24/52

2424

Report suggestions:Test Case Coverage by TCER(Live)

TCER Listing (Live) – with parameters for test casecategory type and name to gather data

Execution Status by Owner using TCER Count (Live)Test Cases (Live) – parameters of Test Case Type andName used to gather data

Test Cases by Team (Live) - parameters of Test Case

Type and Name used to gather data



25/52

Best Practices:Execution


26/52

2626

Topics

Tester best practices

Test lead bestpractices

Test Execution


27/52

2727

Test Execution – Tester best practicesPerforming test execution tasks

As a tester, whattest case executionrecords do I need torun or resume?


28/52

2828

Test Execution – Tester best practicesMonitoring tester execution progress of a suite

A test suite is created, approved and ready for test. It has been scheduled for aniteration. Application is deployed – Tanuj gets started with his testing. Stops for lunch.Restarts for the afternoon after the staff meeting.Pauses for the day. X TCERs completed with 4 defects entered.

Let’s show the best way for Tanuj to get started the next dayLet’s show how Tanuj arranges his personal dashboard to track his progressHow can he decide what TCER to execute nextWhat is status of defectsWhat reports is test lead Tammy running to track progress


29/52

2929

Test Execution – Tester best practicesPerforming test execution tasks

Enable testers to use dashboard viewlets as their execution to do lists by:Creating shared execution record queries that leverage the Current User filtervalue for the Owner column and the Last Result column. Examples:

My Not Run test case execution recordsMy In Progress/Paused test case execution records

Using the Execution Record(s) widget to expose each test case execution

record query to a tester tab in the project dashboard.

Content seen by Tester 1: Content seen by Tester 2:

Total count returned by dashboardviewlet when viewed by Tester 1.

Total count returned by dashboardviewlet when viewed by Tester 2.


30/52

3030

Test Execution – Tester best practicesResuming execution of a suite

Test can be resumed from the testsuite execution record

Resumesuiteexecutionfrom theexecutionconsole oradashboardquery


31/52

3131

Test Execution – Tester best practicesMonitoring tester execution progress of a suite

Option to create TCERs when aTSER is created.


32/52

3232

Test Execution – Test lead bestpracticesMonitoring execution progress

As a test lead, howcan I monitor

execution progressand determinewhere defects arebeing found by myteam?

T t E ti


33/52

3333

Test ExecutionMonitoring execution progress

Tammy, our test lead, needs to create some shared tables and reports to monitorProduct application quality across the iteration as test progresses.She want to know which high priority test cases are passing, failing or blocked andwants to be able to see how the defects entered are progressing to resolved so theycan be verified by her team.What is the best way she can do that?

She will gather the specific test cases she needs by using a pre-defined filter ontest case categories in the execution record table. Gets a lot of records returned!She will then narrow that down by determining the iteration she wants – can bespecific or ‘current iteration’Then she groups by priority or perhaps Owner.She adds a column for defects so she can see resolution status and link to themquicklyShe can drag/drop columns for quick visual editing.What reports can she run to track progress


34/52

3434

Test ExecutionMonitoring Execution Progress

Use test Case Categoriesto narrow down the testcase execution records

Narrowed – but stillmore work to filterneeded.


35/52

3535


Group by Priority to seeprogress bars

Column for defects tosee resolution statusand link to them quickly

Saved query for re-useand sharing


36/52

3636

Report suggestions:

Execution and defects by owner* - Displays the status ofexecution records and their associated defects for eachowner. Data Warehouse report.

Defect list* - Lists defects associated with the selected TestPlan(s) and iterations. Data warehouse report.


For the teamDashboard, a set ofquery links.Reports can be addedto widgets on the

dashboard also


37/52

Advanced Topic:

DevOps


38/52

Running tests as part of your build:a key to devOps success

• The RQM Execution Tool allows for “headless” (unattended) execution ofautomated tests

• Two options:

– Run a standalone java program

– Embed execution directly as an Ant task

• Standard pattern:

– Build

– Provision (uDeploy, Chef, etc)

– Test (automated tests via RQM Execution Tool)

• Tip : use execution variables to parameterize your test

• https://jazz.net/wiki/bin/view/Main/RQMExecutionTool

38


39/52

39

Develop Build (multiple per day)Manual Test (daily but not

completed in a day)

FunctionTest

PerformanceTest

System Test

BuildUnitTest

ALMIntegration

Test

Develop

Staging (daily)

Production

(end of eachsprint)

StagingEnvironment

ProductionEnvironmentIntegration

Test

Rational

Jazz Build

Rational

Collaborative

LifecycleManagement

JUnit

IBM Workload

Deployer

IBM UrbanCode

Deploy

CSPF

IBM UrbanCode DeployIBM Workload

Deployer

IBM UrbanCode Deploy

devOps in ALM


40/52

40

Existing integrations with test automation tools

Quality Manager

Managing mobile applications testingacross a global handset testenvironment

Accelerating test sandbox creationthrough service virtualization

Quick Test ProfessionalLoadRunner

Embedded SW Testing

Rational Integration Tester(GreenHat)

Rational Functional Tester

Rational Performance Tester

Rational Service Tester for SOA

Quality

Worksoft Certify

Rational AppScan Tester Edition

Rational Test RealTimeTelecom Test Automation

VeriStand & TestStand

Selenium Junit tests


41/52

41

Selenium Junit test execution adapter

• Selenium primarily automates web applications for testing purposes

• Support for executing JUnit tests that call the Selenium 2 APIs to drive webbrowsers

• Gather the JUnit results and present them in a test case result

Support developer testing of Web UI


42/52

Advanced Topic:

Command Line Adapter


43/52

Command Line Test Adapter

• Offers a “low barrier to entry” for most of your custom, home

grown, or unsupported testing tools

• Run any test program from the shell and:

• Capture verdict via return code

• Capture standard out and standard error as the execution log that'suploaded to RQM

• Tip: GREAT article - https://jazz.net/library/article/809

43


44/52


• Tip 1: you can map return codes from the command line script to

RQM verdicts using the commandline.properties file

Note: using “*” maps any undefined return to that verdict.

44


45/52


• Tip 2: there is a set of OOTB execution variables that include

most details of the test being run, and you can define customexecution variables

– Passed via the ENV block to the test being run

– Use Project execution variables to define common vars andenumerated values to minimize error and promote reuse!

45


46/52


Did you know you can track progress?

Use com.ibm.rqm.commandline.executionproperties to map afile where you write progress to, and it's visible in the RQM webUI. Cool!

You can periodically update an integer from 0-99 to showpercentage complete

You can add a status message

You can even embed a clickable link, e.g. if your automation

harness has a management console, etc.

46


47/52


• Need a bigger boat?

− You can write your own test automation integration usingthe OSLC Automation specification for RQM

More flexibility, but higher cost of entry compared to thecommand lined adapter

− You can provide custom XHTML to be embedded intoresults, giving more UI flexibility

− https://jazz.net/wiki/bin/view/Main/RQMTestAutomationAdapterAPI

47


48/52

Questions?


49/52

Where are we in the Solution EXPO?

49

Business Process &

Decision Management

Application Infrastructure

& Integration

Mobile Enterprise

Cloud Business

Applications

Asset Management &

Internet of Things

Cloud Infrastructure & IT

Optimization

DevOps & Continuous

Engineering

Security Intelligence

T1 T2

T1 – IBM RTW (integration testing)IBM RTVS (Service Virtualization)

T2 - IBM RTW (functional,performance, & mobile testing)IBM RQM (quality management)

Notices and Disclaimers


50/52

Notices and DisclaimersCopyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced ortransmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or d isclosure restricted by GSA ADP Schedule Contract w ith

IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has beenreviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBMshall have no responsibility to update this information. THIS document is distributed "AS IS" without any warranty, either expressor implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, lossof data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the termsand conditions of the agreements under which they are provided.

Any statements regarding IBM's future d irect ion, intent or product p lans are subject to change or wi thdrawal wi thout

notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples arepresented as illustrations of how those customers have used IBM products and the results they may have achieved. Actualperformance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products,programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do notnecessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neitherintended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legalcounsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’sbusiness and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice orrepresent or warrant that its services or products will ensure that the customer is in compliance with any law.

Notices and Disclaimers (con’t)


51/52

Notices and Disclaimers (con t)

Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products in connection with thispublication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM

products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of thoseproducts. IBM does not warrant the quality of any third-party products, or the ability of any such third-party productsto interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but notlimited to, the implied warranties of merchantability and fitness for a particular purpose.

The provision of the information contained herein is not intended to, and does not, grant any right or license under anyIBM patents, copyrights, trademarks or other intellectual property right.

• IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document

Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand,ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™,PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®,pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®,urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of InternationalBusiness Machines Corporation, registered in many jurisdictions worldwide. Other product and service namesmight be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at"Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtml


52/52

Thank You

Your Feedback isImportant!

Access the InterConnect 2015Conference CONNECT AttendeePortal to complete your sessionsurveys from your smartphone,

laptop or conference kiosk.

rqm best practices

Documents