System z Performance & Capacity Management using TDSz and DB2 Analytics Accelerator: UNIPOL customer experiences

Bruna Murotti,manager of mainframe IT system environment, UNIPOL

Fabio Riva,zStack Advocate, zClient Architect, IBM Italy

Francesco Borrello, Technical Sales, IBM Italy

2

Please note

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be reliedon in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment.

The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed.

Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

3

“After introduction of DB2 Analytics Accelerator it became possible to improve BA and BI solution on System z. One of the many possible exploitations is related to performance and capacity data analysis, a very rich context of structured Big Data to deal with.

In the past, space and database response times limitations restricted the exploitation of this solution. Both of them are now solved, having up to 192 TB of space and 96 parallel cores available with IDAA.

We'll present how it's possible to start from SMF detailed data, collect them with TDSz in a structured way and calculate on the fly performance/SLA enhanced COGNOS reports.

We'll also use SPSS BA tool to develop capacity forecasts based on historical data. Hours of computation will now became minutes, minutes will become seconds. Isn't it an innovative solution?”

System z Performance & Capacity Management using TDSzand DB2 Analytics Accelerator: UNIPOL customer experiences

Abstract

4

Agenda

Introduction – IT capacity management1

Results obtained6

Next steps7

The designed solution for UNIPOL5

Customer needs – Pain points4

Q&A - Closure8

Customer environment3

IBM Capacity Management Analytics for zEnterprise2

5

part 1:

IT Capacity Management

6

Analytics in IT = Capacity Management

Definition from ITIL V3:

– ITIL Capacity Management aims to ensure that the capacity of IT services and

the IT infrastructure is able to deliver the agreed service level targets in a cost

effective and timely manner.

– Capacity Management considers all resources required to deliver the IT service,

and plans for short, medium and long term business requirements.

Sub Processes:

– Component Capacity Management

– Service Capacity Management

– Business Capacity Management

– Capacity Management Reporting

7

Why Capacity Management is important

Helps consolidate and reduce costs

– Reduces HW and labor costs

– Reduces number of physical servers required to run workloads

– Reduce number of required licenses

Helps ensure application availability

– Are any resources overloaded? When will physical resources reach their limits?

– Have there been any significant changes in my environment between two weeks?

– Ensure supply can meet demand

– Ensure business policies are met

Helps optimize resource utilization

– Right size virtual machines

– Identify trends for workload balancing

8

Questions Capacity Management can Answer…..

System/Workload Characteristics, Performance and Trending

• How is my environment performing overall?

– Which are my most used servers/LPARs for a given resource type?

– Are there any bottlenecks in my current environment and where?

– Am I reaching capacity on resources and which resource? When will I exhaust

capacity?

– Which is my top resource consumers for a given resource type?

– Which are my least used servers/LPARs for a given resource type?

– Which are my bottom resource consumers for a given resource type?

– Do I have any outstanding abnormal behavior this week compared to last week

(other periods can be used)?

– Are my systems/workloads balanced or unbalanced?

9

Questions Capacity Management can Answer…..

System/Workload Estimation and Optimization

(optimize and keep optimized – what if)

– How many more VMs can I add to a cluster/server based on usage history?

– How much more resources do I need to add additional VMs to environment?

– How, where do I add capacity if existing systems are not enough for future growth

for optimized capacity usage?

– Where do I place new workloads? Do I really need to add more resources?

– How can I optimize the VM/LPARs placement to maximize usage and minimize

costs?

– How can I optimize the app placement to maximize usage and minimize costs?

10

part 2:

IBM Capacity Management Analytics for zEnterprise

11

Manage the complete time

horizons

System Management

Problem Identification and Resolution

Capacity Forecasting & Real-time Analysis

Historical reporting of past performance

Forecasting future requirements

Real-time transaction monitoring

Jumpstart your time to

value & eases the path to implementation.

Built on IBM’s easy of use analytics

Includes prepackaged, interactive reports

Optional services and education

A single, integrated cost

effective solution

What is IBM Capacity Management Analytics?

It’s everything necessary for the cost effective analysis of zEnterprise usage, service objectives, resource utilization, system tuning, accounting, cost recovery, and more…..

IBM Capacity Management Analytics

12

Capacity Management Analytics: understand how current system is running

System Management

Complete reporting and dashboards

capabilities so all system managers &

executives can view, interact with and

personalize it in ways that support the

unique way they analyze performance

and make decisions

13

Problem Identification and

Resolution

Delivers a top down view of zEnterprise

workloads with the ability to drill into

further detail, perform simple adhoc

analysis to get to the "why", create

system alerts or monitor performance in

near real-time to predict potential issues

before they impact the business.

Capacity Management Analytics : Understand why and how to fix it

14

Capacity Forecasting & Real-time

Analysis

Forecast future capacity to ensure the

capacity is available that the business

needs, when they need it.

Real-time scoring of transactions as they

flow through the system enabling you to

compare with forecast.

Capacity Management Analytics: Build a plan and track against it

15

Pixel perfect reportingA workspace with greater

power, intuitive navigation & cleaner look

Seamlessly shift to more advanced

analysis interaction

Communicate your analysis

using Microsoft Office

Analytics on the go with Mobile devices and disconnected interaction

Advanced Filtering

Built on IBM’s ease of use analytics solution

16

Prebuilt interactive reports and models

MIPS Used -zServer I LPAR Level

Analyze CPU usage by processor type (CP, IFL, zl lP, etc) at the mainframe/CEC

level and identify the LPARs driving the usage.

........ ....

MIPS Used - System Level (Captured vs Uncaptured)

Analyze a system's capture ratio to determine if CPU time consumed by system related

processes (uncaptured CPU time) is too high .

MIPS Used - Service Class Period Level

Analyze the workloads (service classes) driving CPU usage on a system.

Latent Demand

DcPu MIPS Used - zServer I LPAR Level w/Forecast

Analyze future CPU usage based on the results of

the SPSS predictive analytics CPU forecast model.

Determine if latent demand (hidden capacity demand) exists on a system due to the number of tasks wanting to be dispatched exceeds the number

of processors/engines online to a system.

IBM Capacity

Management Analytics

WLM (Workload Manager)

Delays by Importance Level

Analyze the types of delays impacting each WLM

importance level (highest importance to lowest importance). Is your most impor tant work

being negatively impacted by delays?

Delays by Service Class Period

Analyze the types of delays impacting each WLM service

class period. Which service class periods assigned to an importance level are being negatively impacted by delays?

Model

Solution Kit (Prebuilt interactive

reports and models)

LPAR CPU Forecast

SPSS predictive analytics model that forecasts LPAR CPU usuage

at the hour, day and month levels

CSA/ECSA/SQA/

ESQA Utilization

Analyze peak/max utilization for the common virtual storage

areas: CSA, ECSA, SQA & ESQA.

Unplanned system outages can occur when available CSA

or ECSA storage is exhausted.

........ ...

Performance Indexes

Analyze how well Workload Manager is doing with goal achievement. How often are WLM

goals being met (Pl <= 1) or missed (Pl > 1)?

Pulse2014 The Premier Cloud Conference

17

Optional: SCCM Optional: Distributed data feed

IBM CMA core architecture diagram

18

Cognos Business Intelligence

provides the range of analysis capabilities necessary for optimizing zEnterprise use by

confidently and simply compiling the information necessary to understand and manage

system activity while significantly improving the ability to identify potential issues and

pinpoint their cause.

A look under the covers: IBM Cognos BI on zEnterprise

19

A look under the covers: IBM SPSS Modeler for Linux on zEnterprise

SPSS Modeler with Scoring Adapter

can help you use predictive analytics to

forecast future requirements for zEnterprise

and ensure the capacity required is available

when the business needs it. The Scoring

Adapter provides real-time scoring of transactions as they flow through the

system enabling you to compare actual

usage with expected and identify anomalies

before they can adversely affect the system.

20

A look under the covers: Tivoli Decision Support for z/OS

Tivoli Decision Support for z/OS

enables the data collection for the solution and builds the capacity warehouse in DB2 for

z/OS that Cognos Business Intelligence and SPSS Modeler access for reporting, analysis

and predictive modeling. Tivoli Decision Support for z/OS is also able to collect capacity and

performance data for virtually all platforms that are used in business today.

Performance

Chargeback & Accounting-

Cost Recovery

Service Level Reporting

21

IBM DB2 Analytics Accelerator : Providing faster analysis of your capacity requirements!

• What does it do?

– Base forecasts off larger samples of historical SMF data to improve accuracy of predictive models

– Dramatically accelerate the analysis of your zEnterprise usage & performance data

– Significantly speed up complex queries of the large volumes of data that are being created by zEnterprise.

– Lower the cost of long-term storage of large volumes of historical SMF data with a high-performance storage saver feature

IBM DB2 Analytics Accelerator

� What is it?

• A high performance appliance that speeds analysis, enabling you to base your projections on a larger sample of historical data

22

IBM DB2 Analytics Accelerator : Query Execution Flow

DB2 for z/OS

Optimizer

IDA

A D

RD

A R

equesto

r

IBM DB2 Analytics Accelerator

Application

Application

Interface

Queries executed with DB2 Analytics Accelerator

Queries executed without DB2 Analytics Accelerator

Heartbeat (DB2 Analytics Accelerator availability and performance indicators)

Query execution run-time for

queries that cannot be or should

not be off-loaded to IDAA

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SM

P H

ost

Heartbeat

Faster Answers, Faster Reports

23

IBM DB2 Analytics Accelerator : High Performance Storage Saver

Reducing the cost of high speed storage

• Time-partitioned tables where:

– only the recent partitions are used in a transactional context (frequent data changes, short running queries)

– the entire table is used for analytics (data intensive, complex

queries).

• DB2 partitions are deleted after the High Performance Storage Saver are created on the accelerator

DB2

#1

Accelerator

#1

Query from

Application

Or

Accelerator

#2

Accelerator

#3

Accelerator

#4

Accelerator

#5

Accelerator

#6

Accelerator

#7

No longer present on DB2 Storage

24

part 3:

The Customer environment

25

UNIPOL Hardware Technical Environment

2 IBM 2827-H20 (707 + 704)

Production :

o 7 GP + 3 zIIP + 6 zAAP + 1 ICF

o 384 GB Memory

o 1092 MSU – 8954 MIPS

Development :

o 4 GP + 1 zIIP + 3 zAAP + 1 ICF

o 384 GB Memory

o 664 MSU – 5409 MIPS

Appliance:

o DB2 Analytics Accelerator for z/OS

Storage MGM Configuration

o DS8870 + DS8800 MM (Sync)

o DS8800 + DS8700 GM (Async)

o 2 X (TS7720 + TS7680) Virtual + TS3500 Real

14363MIPS

14363MIPS

1756MSU

1756MSU

26

UNIPOL Hardware Technical Environment

Disaster Recovery site

IBM 2817- M15

o 7 GP processors

o 1 zIIP SE

o 5 zAAP SEs

o 1 ICF

o 165 GB Memory

Three sites configuration

27

UNIPOL Mainframe Technical Environment

z/OS version 1.12

DB2 version 10 NFM

6 Subsystems

CICS TS 4.2

50 Subsystems

9 million transactions/daily

WAS 7.0 on z/OS

11 Application Servers (2 clustered)

6.5 million threads/daily

WebSphere MQ 7.0.1

28

UNIPOL Application Environment

• Cobol cics/batch – static and dynamic

• Assembler

• JAVA (SQLJ/JDBC)

• DELPHI (ODBC) on Workstation

• Visual Basic - .NET on Workstation

29

part 4:

The Customer needs – pain points

30

• Customer needed a solution to control all resources required to deliver the IT service, and plans for short, medium and long term business requirements

• Solution should follow cost reduction directive, so the consumption of MIPS and use of storage should be reduced and kept as small as possible

• Improvement in the existing user interface should be provided, allowing intuitive navigation and easy-to-use tools

31

part 5:

The designed solution for UNIPOL

32

An overview of UNIPOL solution architecture

DB2

TDSz

SMF logs

Cognos

SPSS

Most recent data (2-8 days) <= 1TB

NNN weeksstored in HPSS

Reports

Most recent data(2-8 days)

+

Physical 16 TB(64 TB uncompressed)

33

The plan to realize the designed solution

• Phase 1: upgrade & reports•environment upgrade (TDSz new features) and setup of Cognos report environment

• Phase 2: speed up & archives:•setup and test of IDAA environment•test of IDAA queries•tables partitioning and archives with data compression

•Phase 3: ideas for new functions – to be verified and discussed•use only TDSz detailed data with IDAA aggregations•use of LOAD instead of INSERT in order to use Turboloader•SPSS for forecasting

34

part 6:

Results obtained

35

Phase 1: upgrade & reports

• TDSz: upgrade maintenance and installation of CICS feature

– maintenance update of PTS

– Installation of CICS feature

• COGNOS: setup and connection to TDSz DB2 database

– Migration of existing reports and definition of new ones

• installation of TDSz provided reports

• migration of existing reports to COGNOS

• development of new local reports to satisfy new requirements

• definition of users group in order to control access to database

– Reports scheduling and automatic distribution inside UNIPOL

• schedule of predefined reports with output in PDF format

• distribution of PDF reports to defined users

• output in PDF with graphs format or columnar data

36

Phase 2: speed up & archives 1/2

• Identification of environment to test solution

– Test environment with TDSz database or directly in production?

• Identification of TDSz queries

– Analysis on existing elapsed time/consumption data to define the list of queries

to be used for benchmarking

• TDSz queries measurements

– measurement forcing DB2 execution

• SET CURRENT QUERY ACCELERATION NONE;

– measurement forcing IDAA execution

• SET CURRENT QUERY ACCELERATION ALL;

37

Phase 2: speed up & archives 2/2

• Identification of TDSz tables for partitioning

– analysis based on TDSz queries

– list of tables to be modified

• Partitioning tables (partitioning based on time criteria)

– Alter + Reorg DB2 commands

– archiving tables

• Force the use of IDAA

– reduction of MIPS usage

– access to all data in tables

– we force the use of IDAA from the queries instead of forcing it from DB2, so DB2

administrator can globally select the databases to be under IDAA optimization

38

Queries on TDSz data: values we detected (Montpellier lab environment)

sed time Seconds CPU time from zOS

DB2+1DAA difference 082 082+10AA difference

08209 1,1 -99,9% 1.671 0,02 -99,999%

08210 1,1 -99,9% 1.660 0,02 -99,999%

08215 1 1 -99,5% 192 0,02 -99,990%

08216 3, 1 -99,8% 1.427 0,02 -99,999%

08217 1,1 -95,0% 19 0,02 -99,896%

08218 21 -99,9% 1.609 0,02 -99,999%

08216 3, 1 -99,9% 2.062 0,02 -99,999%

avera e 1.286 1,8 -99,9% 1.234 0,02 -99,998%

10000 i-----------------------::=====:::1

1000

100

10

DB209 DB210 DB215 DB216 DB217 DB218 DB216

• DB2

oDB2+1DAA

average

Pulse 2014 The Premier Cloud Conference 0

39

Space usage: compression rate we detected (Montpellier lab environment)

Using the average compression rate

5,69

A full rack model 2001

will store 273 TB

40

part 7:

Next steps

41

Next steps: ideas for the future we’re working on

• Use of IDAA v4

– performance improvements

– improvements in archive operation (automation of manual activities)

• Keep only detailed data in TDSz tables.

– All the depending data will be calculated on the fly by IDAA and only detailed data will be stored in IDAA storage

• Use of SPSS to forecast resources requests

– SPSS can forecast future capacity to ensure the capacity is available that the business needs, when they need it.

– Real-time scoring of transactions as they flow through the system will enable us to compare with forecast.

• Evaluate with TDSz labs the possibility to use LOAD function instead of INSERT

– This in order to have performance data only on IDAA

42

part 8:

Q & A

43

© Copyright IBM Corporation 2014 All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Cognos, the Cognos logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

Thanks for your attention!

44

© Copyright IBM Corporation 2012. All rights reserved.

– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract

with IBM Corp.

– Please update paragraph below for the particular product or family brand trademarks you mention such as WebSphere, DB2,

Maximo, Clearcase, Lotus, etc

IBM, the IBM logo, ibm.com, [IBM Brand, if trademarked], and [IBM Product, if trademarked] are trademarks or registered trademarks of

International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are

marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common

law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law

trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at

www.ibm.com/legal/copytrade.shtml

f you have mentioned trademarks that are not from IBM, please update and add the following lines:

[Insert any special 3rd party trademark names/attributions here]

Other company, product, or service names may be trademarks or service marks of others.

Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all

countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are

provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice

to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is

provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of,

or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the

effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the

applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may

have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these

materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific

sales, revenue growth or other results.

Acknowledgements and Disclaimers: