stop feeding ibm i performance hogs - robot
TRANSCRIPT
![Page 1: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/1.jpg)
All trademarks and registered trademarks are the property of their respective owners.© HelpSystems LLC. All rights reserved.
How Robot Monitor can help you pen up runaway processes before they cause problems
Stop Feeding IBM iPerformance Hogs
![Page 2: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/2.jpg)
HelpSystems LLC. All rights reserved.
YOUR HOSTS
Chuck LosinskiDirector of Automation Technology
HelpSystems
Broadcasting live from Eden Prairie, Minnesota USA
and Düsseldorf, Germany
Kurt ThomasSystem Engineer
HelpSystems
![Page 3: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/3.jpg)
HelpSystems LLC. All rights reserved.
• Resource-hogging scenarios and jobs
• Qualify impact on CPU,
memory, and storage
• Identify and address resource
hogs automatically
AGENDA
![Page 4: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/4.jpg)
HelpSystems LLC. All rights reserved.
MOST COMMON HOGS
What types are most common?
• CPU hogs
• Main storage hogs
• Temporary storage hogs
How can you locate them and take action?
• A reactive solution: WRKACTJOB, WRKSYSACT, iNAV
• A proactive solution: Robot Monitor
These impact job run time
![Page 5: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/5.jpg)
HelpSystems LLC. All rights reserved.
Why are these issues so problematic to an IBM i environment?
• Rapid resource consumption
• Need to be found quicklyResource
Hogs
• Limited
• ExpensiveResources
PROBLEM PROBLEM PROBLEM
Userimpact
Demandfor moreresources
Systemfailure
CHALLENGES
![Page 6: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/6.jpg)
UP NEXT...
CPU Hogs
![Page 7: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/7.jpg)
HelpSystems LLC. All rights reserved.
What is the impact on your enterprise?
System symptoms:
• CPU is being rapidly consumed
Impact:
• Until the job(s) causing
this are identified and
the issue resolved,
expect:
o Additional resource expenditure
o Impairment of user productivity
o Impact on many systems
o POS, web, DB, GUI, credit cards
Rapid CPU consumption
Operators in race against the clock to identify
CPU hogs
System productivity is maintained onlyby feeding extra
resources
All of which consumes more
resources
Underlying issue remains until the
team runs outof resources or
resolves it!
CPU HOGS
![Page 8: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/8.jpg)
HelpSystems LLC. All rights reserved.
QZDASOINIT and QSQSRVR Jobs
What are they?
• DB2 database server jobs
What do they do?
• Let applications pull information from, and modify, information in DB2 on IBM i
• QZDASOINIT jobs implement ODBC, JDBC
• QSQSRVR implements Server Mode = IBM-flavor ODBC
Why so problematic?
• The issue with QZDASOINIT and QSQSRVR jobs is not the jobs themselves—
it is poorly-written SQL code running in them that causes the issues.
CPU HOGS
![Page 9: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/9.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSHow to find the maximum number of QZDASOINIT jobs in QUSRWRK
Step 1: Run DSPSBSD SBSD(QUSRWRK)
![Page 10: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/10.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSHow to find the maximum number of QZDASOINIT jobs in QUSRWRK
Step 2: Select Option 10 (Display Prestart Job Entries)
![Page 11: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/11.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSHow to find the maximum number of QZDASOINIT jobs in QUSRWRK
Step 3: Select Option 5 (Display Details on QZDASOINIT)
Result: The maximum number of jobs and uses
![Page 12: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/12.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSHow to display Active Prestart Jobs
Step 1: Run DSPACTPJ SBS(QUSRWRK) PGM(QZDASOINIT)
Result: Shows the current number of prestart jobs and how many are still in use
![Page 13: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/13.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSHow do we find the offending job FAST?
Common example:
• QZDASOINIT jobs
Challenges to identification:
• Jobs all share the same name
• Potentially hundreds of jobs could be contributing to the issue
• The longer this process takes, the more CPU is consumed and the greater
the risks and impact on resources
![Page 14: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/14.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSReactive approach
Step 1: WRKACTJOB followed by manual batch investigation and resolution at
the job level on each LPAR you manage
![Page 15: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/15.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSTroubleshooting
Big picture questions:
• Who is running these jobs?
• What proportion of overall CPU are these jobs consuming?
Conclusion:
• Greater insight will provide deeper understanding and context to any issues,
which will result in faster problem resolution.
![Page 16: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/16.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSProactive approach
• Step 1: Real-time visibility
• Step 2: Immediate access to jobs for resolution and threshold levels
for early detection
![Page 17: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/17.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSProactive approach
Data definitions: Define by any/all systems + Define by job name
![Page 18: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/18.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSProactive approach
Thresholds: Custom thresholds and proactive alerts
![Page 19: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/19.jpg)
HelpSystems LLC. All rights reserved.
CPU HOGSProactive approach
Monitored groups: Granular monitoring can extend to subsystem, accounting
code, user, current user, job, and function
![Page 20: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/20.jpg)
HelpSystems LLC. All rights reserved.
• Identify and manage CPU hogs
LIVE DEMO
![Page 21: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/21.jpg)
UP NEXT...
Main Storage Hogs
![Page 22: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/22.jpg)
HelpSystems LLC. All rights reserved.
Page faults
System symptoms:
• The number of paging faults rapidly increases, a condition known as “thrashing”
Impact:
• Until the job(s) causing this
are identified and the issue
resolved, expect:
o Number of page faults
to rapidly rise
o Jobs to take longer
and longer to execute
o Impairment of user productivity
Data loaded/re-loaded into
memory
Batch job beginsto process
Interactive job flushes memory
records
Batch job can’t access the
records (faulting occurs) and...
ThrashingCycle
MAIN STORAGE HOGS (RAM)
![Page 23: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/23.jpg)
HelpSystems LLC. All rights reserved.
Reactive approach
Step 1: Access the Work with System Status screen to show number of page
faults in each memory pool
MAIN STORAGE HOGS
![Page 24: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/24.jpg)
HelpSystems LLC. All rights reserved.
Troubleshooting
Big picture questions:
• Which jobs are responsible for causing problems in these memory pools?
• Which subsystem(s) are using these memory pools?
Conclusion:
• Greater insight will give more meaningful understanding and context to any
issues for faster problem resolution.
MAIN STORAGE HOGS
![Page 25: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/25.jpg)
HelpSystems LLC. All rights reserved.
Proactive approach
Step 1: Real-time visibility of dedicated NDB bar shows us the overall system
faults/second.
Stage 2: Immediate access to offending jobs for resolution. Threshold levels
for early detection.
MAIN STORAGE HOGS
![Page 26: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/26.jpg)
HelpSystems LLC. All rights reserved.
• Identify and manage main storage hogs (thrashing)
LIVE DEMO
![Page 27: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/27.jpg)
UP NEXT...
Disk Hogs
![Page 28: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/28.jpg)
HelpSystems LLC. All rights reserved.
What is temporary storage?
• Storage allocated by the OS in the system ASP (disk). Its contents will be
cleaned up when an IPL is performed.
• Programs loaded into the activation group and the associated variables, heap space (Java as well as other HLL APIs, i.e., malloc, calloc), open data paths, etc.
Temporary storage?
YES!
• Objects in QTEMP librariesTemporary storage?
NO!
DISK HOGSTemporary storage
![Page 29: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/29.jpg)
HelpSystems LLC. All rights reserved.
DISK HOGS
System symptoms:
• Overall system DASD is rapidly being consumed as a result of underlying temporary
storage being used by a runaway job
Impact:
• Until the job(s) causing this are identified and the issue resolved, expect:
o Additional disk requirement
o Potential system failure if left unchecked
Runaway job
Temporary storage
Overall DASD
Temporary storage
![Page 30: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/30.jpg)
HelpSystems LLC. All rights reserved.
Common example:
• Java memory leak—Java job allocates memory to run tasks, but never
returns memory to OS after finishing task
Challenges to identification:
• Where is storage being used?
• Which jobs are consuming temporary storage?
DISK HOGSTemporary storage
![Page 31: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/31.jpg)
HelpSystems LLC. All rights reserved.
Step 1: Use the command WRKSYSSTS to determine 2 values:
• Current temporary storage consumption
• Peak temporary storage consumption
DISK HOGSReactive approach
![Page 32: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/32.jpg)
HelpSystems LLC. All rights reserved.
Step 2: Run the WRKSYSACT command as follows: WRKSYSACT
SEQ(*STGNET) to determine which jobs are using the most temporary storage
• Only shows jobs that have had activity since the last collection
• Can also include permanent storage
DISK HOGSReactive approach
![Page 33: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/33.jpg)
HelpSystems LLC. All rights reserved.
Step 1:
• Real-time visibility of the temporary storage being used by each subsystem
Step 2:
• Immediate access
to offending jobs
for resolution
• Threshold levels
for early detection
DISK HOGSProactive approach
![Page 34: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/34.jpg)
HelpSystems LLC. All rights reserved.
• Identify and manage temporary storage hogs
LIVE DEMO
![Page 35: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/35.jpg)
HelpSystems LLC. All rights reserved.
• Resource-hogging scenarios and jobs
– How to identify them
• Qualify impact on CPU,
memory, and storage
– How this will affect users and applications
• Identify and address resource
hogs automatically
– Thresholds and notification
RECAP and Q&A
![Page 36: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/36.jpg)
HelpSystems LLC. All rights reserved.
Upcoming Performance Monitoring Webinars
• SQL-Based Monitoring
– September 7
• Real-Time Disk Monitoring
– September 28
Integrating Robot Monitor into your infrastructure
![Page 37: Stop Feeding IBM i Performance Hogs - Robot](https://reader031.vdocuments.site/reader031/viewer/2022022415/5a6ee9117f8b9a4f608b51f5/html5/thumbnails/37.jpg)
HelpSystems LLC. All rights reserved.
Thank you for attending!
Contact Information
Website:
www.helpsystems.com/robot
Telephone:
800-328-1000 sales
+1 952-933-0609 support
Presenters: