z Technical

DevOps: Continuous Integration for z Using Test Data Management and Application Development Foundation Using IBM Rational developer & test with RDz and Optim TDM Solution Keith Allen – European System Z DevOps BUE Frank Jones – WW Systems Z Certified Consulting IT Specialist

Please Note:

1

• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in

making a purchasing decision. • The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any

material, code or functionality. Information about potential future products may not be incorporated into any contract. • The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. • Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual

throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Cont Int

Agile Dev

What is DevOps?

2

Addressed by... Addressed by... Addressed by...

34% of all new IT Projects deploy late*

*Internal surveys, and commissioned studies

experience delays in integration, configuration and

testing of applications

41% experience delays

due to troubleshooting and fine-tuning issues

in production

45% applications rolled back due to quality

issues escaping into production

51%

to deliver a simple change

Up to

4-6 Weeks

Software Development

Test Operations Line of Business

GAP GAP GAP

DevOps is a Journey:

Cont Deploy

Agenda

• What is Continuous Integration Testing for z • Why Continuous Integration for z • How can you implement Continuous Integration for z • ‘Day in the Life of a Developer’ Demonstration • Q & A

4

Continuous Integration Testing Solution for System z (CIz)

The continuous integration testing process consists of: 1. Integrating Testing Practices at the development stage.

2. Continuously compiling, inspecting, building, and invoking

automated testing of source code changes.

3. Finding defects earlier in the AD cycle. Not reliant on manual testing.

‘Expedite feedback to developers on application quality’

5

During the CODING phase

$80/defect

During the BUILD phase

$240/defect

During the QA/TESTING

phase

$960/defect

Once released as a product

$7,600/defect +

Law suits, loss of customer trust, damage to brand

80% of development costs are spent identifying and correcting defects!*

*National Institute of Standards & Technology

Source: GBS Industry standard study

Defect cost derived in assuming it takes 8 hrs to f ind, f ix and repair a defect w hen found in code and unit test. Defect FFR cost for other phases calculated by using the multiplier on a blended rate of $80/hr.

If admitted or not most development LPARs are managed as if starting here

WASTE = Feedback to Developers

Desired state: Ensure Automated Testing at the Development and Build stage

Contributing Factors to Poor Software Costs

How to Implement System z CIz Application Testing Solution 1. Provide a mechanism to deploy Dev/Test environments

quickly in an agile way

2. Provide a mechanism to enable developers to run automated tests to provide early feedback on application quality

– Reports can be fed to a Management dashboard for approval/review

6

Application Testing Today

7

Development Environment Code

Check In Build

Developer Tester

Promote To QA

Feedback

Change Request

Open Defects

Queue Request Deploy

Queue Request

Run Tests

System Programmer

But… It takes days or even weeks to test and fix changes due to reliance on

(1)manual processes and (2)limited access to limited test environments!

System z CIz Application Testing Solution Now

8

Development Environment

Developer

Rapid Feedback on Quality of

Code for Developers and

Management

Rapid Feedback

Check out

Code from MF Move to Sandbox

Build/Test + Run Automated Tests +

Change Request

Dedicated z/OS

(Sandbox) Environments on

demand for Developer or Project

Automated Testing Processes

Promote To QA

Higher quality going into

QA

Lower cost of application testing using off-mainframe z/OS test environment Fast, dependable, automatic feedback speeds time to market Enables confidence by automatically tracking and promoting code health

Developers control Deployment of Test Data

Insert Test Data

Rational Team

Concert

Rational Developer

for z Systems

Rational Development

and Test

Optim Test Data

Manager

Rational Developer

for z Systems

Rational Developer

for z Systems

Real-time syntax

checking, code

assistance and more

Automated Testing e.g.

Code Review , Code

Coverage and more

Eclipse IDE suitable for

both traditional and

new developers

Integration w ith

mainframe subsystems and Rational

tooling

Rational Developer for System z Overview

Rational Development

and Test

Mainf rame running on x86 f or Dev /Test

purposes only

Real z/OS, Compilers,

Sof tware and Middleware

Dev elopment Sandbox to test

changes in isolation, like Distributed Dev elopers

Agile way to spin up

Dev /Test env ironments

quickly

Rational Development and Test Overview

Rational Team

Concert

Store all project related

artef acts, e.g. Release plans,

work items, story points and

more

Real-time dashboards and

reporting

ISPF, .NET, Eclipse, Web

User Interf aces

Traceability Integrate with

Business Requirements,

Test Plans, Designs etc

Rational Team Concert Overview

Optim Test Data

Management

Data Masking

Extract all / subset of

data

Deploy Test Data

Reusable Refresh

Process for Users/DBAs

Optim Test Data Management Overview

Ciz Demo Architecture

RD&T Env.

Application 1

RD&T Env.

Application 2

Catalogue

ESX

Test Data

Management Server,

Designer & Web UI

RD&T Env.

Application 2

RTC Server

Dashboards

Compile Build Automated Tests

Feed Reports to Management

PROD LPAR

TEST LPAR

DEV LPAR

3rd Party Production SCM

DEV

TEST PRD Clone

On Request

Check Out /In Move to Sandbox

Developer can build on RD&T using RDz generate JCL capability. Developer can trigger off Data refresh/Insert using Optim Web UI Production SCM still maintains single source

Rational Developer for z Systems

Other Continuous Integration for z Architecture

RD&T Env.

Application 1

RD&T Env.

Application 2

Catalogue

ESX

Test Data

Management Server,

Designer & Web UI

RD&T Env.

Application 2

PROD LPAR

TEST LPAR

DEV LPAR

RTC Server / SCM / Build

DEV

TEST PRD Clone

On Request

Check out /in and build + Automated Test on RD&T sandbox

RTC for build, SCM, project related artefacts, release plans and more. RTC can automatically move code to RD&T environments during the build process and run automated tests as part of the Build. Frequent batch process to move baseline

Rational Developer for z Systems

Demo Scenario

Development Environment

RDz Developer

Testing reports are generated

and uploaded to RTC Dashboard

Rapid Feedback

Check out

Code from MF Move to Sandbox

Build/Test + Run Automated Tests +

Change Request

z/OS Sandbox (RD&T) environment

is cloned from Master RD&T Image

Project manager has assigned a RTC Work Item for the Developer to move, build and run

automated tests on their Sandbox

Promote To QA

Move code to Production SCM

and promote

Developers requests reusable data insert service for Dev/Test

activities

Insert Test Data

Developer checks out code from SCM and moves code

to sandbox

1

2 3

4

5

6

Role: Developer

1

2

3

4

RTC Server task

Check out code from Production SCM and move code to RD&T server

Make code change, Build and Test Code in RDz

Run Automated Tests and Feed Results to RTC Server

Demo 1 – Mainframe Development Requirement

Developer is assigned a RTC work item.

The task is to check out from SCM and compile,

build and run automated tests on his RD&T

server. Results need to be fed to RTC Server for management approval

before check in code to Production SCM.

Storing *Master* Images on RD&T

Role: Operations (Test Data Manager)

1

2

3

4

Source Production Data

Desensitize Data

Compare Desensitized Data with Source Data

Create an Insert/Refresh Process for the Developer to use via UI

Demo 2 – Extract Test Data from Mainframe

Using Optim Test Data Management, the

objective is to extract test data from a

source mainframe DB2 environment.

Create a reusable process so the

developer can trigger off the insert of data

via the Web UI.

Role: Developer

1

2

3

4

Update from RTC Server

Triggers off Test Data Insert to RD&T Server

Runs Program to Ensure Code is Working After the Change.

Runs Automated Tests and Feeds Reports to RTC Server for Review

Demo 3 – Developer Resumes Work

Using Optim Test Data Management, the

objective is for the developer to be able

to provision prepared data to their test

database(s).

A reusable process has been created so the developer can

trigger off the insert of data via the Web UI.

Role: Project

Manager

1

2

3

4

Open Completed Work Item

View Test Reports

Approve or Reject

Confirm Completion and Advise Code to be Checked in to Production SCM

Demo 4 – Project Manager Review

Project Manager reviews the work completed by the

developer.

Once approved, the Project Manager

advises the developer to check in code to Production SCM.

DEV Benefits and Value

• Developers/Projects can own and manage pre-defined test environments which can be cloned and deployed for development/testing activities

• RDz eclipse IDE can help boast productivity and provide a mechanism to move code from the mainframe to a RD&T environment(s) – RDz Menu Manager – RTC + Urban Code Deploy

• Rational Team Concert gives management a view of the development work items and a process to ensure automated tests have been completed successfully

• Automated tests are built into RDz and can be centrally managed on the mainframe (including Property Groups) – This includes Coding Standard Rules and Code Coverage Thresholds

DEMO https://www.youtube.com/watch?v=meU_TvNL9qE&feature=youtu.be

OPS Benefits and Value

• Data can be refreshed by developers on demand • Data sourced from ‘production’ can be obfuscated (masked) while maintaining

data integrity and contextual content • Provisioning of data can be monitored and audited as required. • Data sources on both distributed and mainframe (z/OS) platforms are

supported. • Federated data extracts from multiple sources/platforms are supported

• Rational Developer for System z – http://www-03.ibm.com/software/products/en/rational-developer-for-z-systems

• Rational Development and Test Environment – http://www-03.ibm.com/software/products/en/ratideveandtestenviforsystz – RD&T Cloud Managed Service

• Rational Team Concert – https://jazz.net/help-

dev/clm/index.jsp?re=1&topic=/com.ibm.rational.rtc.rdz.doc/topics/rdzrtcz_int_overview.html&scope=null

– https://jazz.net/products/rational-team-concert/ • DevOps z Mainframe Article

– https://www.ibm.com/developerworks/community/blogs/invisiblethread/entry/devops_for_the_mainframe?lang=en

• Trial usage of DevOps software (not including Optim TDM)

24

Information Resources – Rational Testing solutions

Information Resources – Optim Test Data Management solutions

• Optim Test Data Management solution – – General solution information http://www-03.ibm.com/software/products/en/infosphere-optim-test-data-management

– What’s new https://www-01.ibm.com/software/data/lifecycle-management/whats-new.html

Data Privacy elements – General solution information http://www-03.ibm.com/software/products/en/infosphere-optim-data-privacy General solution information

- White papers https://www-01.ibm.com/software/data/optim/protect-data-privacy/resources.html

•Any Questions

Notices and Disclaimers

27

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Notices and Disclaimers Con’t.

28

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank You Your Feedback is Important!

Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your

smartphone, laptop or conference kiosk.