mxg tools and usage
DESCRIPTION
MXG Tools and Usage. Chuck Hopf. Agenda – Building the PDB . VMXGALOC\VGETALOC UTILBLDP BLDSMPDB UTILWORK READDB2 VMXGSUM. Agenda - Analysis. ANALID ANALGRID VMXGPRNT VMXGFIND VMXGSRCH ANALCNCR ANALCAPD. VMXGALOC – Pseudo GDGs. ASCII ONLY – Windows or LINUX - PowerPoint PPT PresentationTRANSCRIPT
MXG Tools and Usage
Chuck Hopf
Agenda – Building the PDB
VMXGALOC\VGETALOC UTILBLDP BLDSMPDB UTILWORK READDB2 VMXGSUM
Agenda - Analysis
ANALID ANALGRID VMXGPRNT VMXGFIND VMXGSRCH ANALCNCR ANALCAPD
VMXGALOC – Pseudo GDGs
ASCII ONLY – Windows or LINUX Allocates directories and assigns
LIBNAMEs using a date based structure Allows you to ‘keep’ as many
generations as you wish of each type of data – daily, weekly, trend, spin, db2acct, cicstran, monthly
VMXGALOC – Parameters
BASEDIR=C:\MXG – where do you want to put the directories? Can be any valid location so long as it is connected to the system executing SAS/MXG
FORCEDAY= used in the event of a rerun or the need to report for some given day - can be a SAS date value – 27AUG12 or a relative value – today()-2
VMXGALOC - Parameters
WEEKSTRT=MON – the day of the week on which your week starts. MON is the MXG default
Number of generations -– WEEKKEEP=12 – keep 12 weeks– DAYSKEEP=14 – keep 14 days– MNTHKEEP=15 – keep 15 months– CICSKEEP=15 – keep 15 days of CICSTRAN– DB2KEEP=14 – keep 14 days of DB2ACCT
VMXGALOC Parameters
RUNWTD=NO – change to yes to run week to date logic
RUNMTD=NO – change to yes to run month to date logic
TRENDING=daily or weekly – how often to update TREND databases
READONLY=yes/no if NO the aging of old generations is suppressed
CLEARALL=YES clears the normal default LIBNAMEs from AUTOEXEC
VMXGALOC Parameters
DATEFMT= can be any valid DATE format…– date date7 date9– mmddyy6 8 10– ddmmyy 6 8 10– yymmdd 6 8 10– julian 5 7
If the format (mmddyy8. for example) contains / then the equivalent mmddyyd8. is substituted
An invalid datefmt will result in an ABEND
VGETALOC
VGETALOC will fetch a ‘range’ of dates for daily/weekly/monthly PDBs and pass that information to VMXGSET so that you could say something like:– %vgetaloc(getdaterange=12jul12 23jul12,
typeofdata=daily,basedir=c:\mxg); data jobs; set %vmxgset(dataset=jobs);
VGETALOC
Can only be used on ASCII systems where VMXGALOC has been used to create pseudo-GDGs
If a date in the date range does not exist it is skipped
VGETALOC - Parameters
GETDATERANGE – the range of dates in the form of SAS date values to be searched
TYPEOFDATA – DAILY WEEKLY MONTHLY?
DATEFORMAT – the DATE format used in VMXGALOC
BASEDIR – the directory as specified in VMXGALOC
UTILBLDP Normally the code to read an SMF record is:
– %INCLUDE SOURCLIB(TYPE30); And to read two types you might code:
– %INCLUDE SOURCLIB(TYPE30);– %INCLUDE SOURCLIB(TYPE1415);
But that would cause two passes of the SMF dataset which can be very large and make this an expensive and time consuming process.
With UTILBLDP this becomes:– %UTILBLDP(USERADD=30 1415,
BUILDPDB=NO,SORTOUT=NO,OUTFILE=INSTREAM);– %INCLUDE INSTREAM;
UTILBLDP
UTILBLDP is a macro designed to simplify adding records to the normal MXG PDB (performance data base.) The coding in exits is not difficult if you understand it all but can be arcane to the uninitiated.
It can also be used to read multiple kinds of SMF data in a single pass of the SMF data and create the SAS datasets in WORK or in a PDB.
UTILBLDP
For documentation on all parameters and usage see the member in the MXG SOURCLIB
For our purposes there are only a few important parameters
SORTOUT=NO – suppresses sorting and writing of the data to the PDB DD. You may want to use the sort (just add a PDB DD to your JCL) as it will remove any duplicate records.
USERADD= a list of the record types you wish to read – 30 6 1415 64 70 etc.
UTILBLDP
OUTFILE= INSTREAM writes the data to the temporary dataset defined by the INSTREAM DD. You can then simply %INCLUDE INSTREAM to execute the code. If you want to STORE the code for future use (or just to see what the generated code looks like) route to a PDB member or a sequential dataset.
BUILDPDB=NO – suppresses the logic that builds the full MXG PDB.
BLDSMPDB
Build the daily/weekly/monthly/trend databases from a single job on ASCII platforms (the JCL just would not work on zOS – could be done using DYNALLOC and LIBNAME statements but that would preclude the use of GDGs.)
BLDSMPDB
There are numerous parameters – too many to mention here but all are documented in the member of SOURCLIB
– Allows for reruns– User code– Run daily/weekly/monthly– Run WTD MTD– Run TRENDing daily/weekly– Read DCOLLECT and Tape management data– And much much more…–
Usage
Combine these to tailor your PDB Use UTILBLDP to add/subtract record
types and specify things to run after BUILDPDB
Use BLDSMPDB to control the execution of BUILDPDB
Example
Suppress CICSTRAN and DB2ACCT but process statistics datasets for both CICS and DB2
Add TYPE6156 and TYPE42 data to the PDB
Suppress TYPE74 data
ANALID
New MACRO to create an SMF Audit dataset and report– READSMF=NO– PRINT=YES– PDBOUT=PDB– PERCENTS=YES– ODS parameters
ANALID – READSMF
READSMF=YES will read an SMF dataset. The default of NO is used in BUILDPDB to read the ID dataset already being created.
Driven by the value of the SMFAUDIT macro variable in VMXGINIT. If set to NO with a %LET the older style report is created with fewer variables.
ANALID – PRINT/PDBOUT/PERCENTS
PRINT=YES – prints SMF Audit report. To suppress the report specify NO.
PDBOUT=PDB – the destination of the new SMFRECNT dataset.
PERCENTS=YES – calculates the percentage of the data for each system represented by a single type/subtype.
ANALID – ODS Parameters
ODSTYPE= if you want to create HTML output specify HTML or specify some other valid ODS value. If blank ODS is not used.
ODSPATH= the pathname for the ODS output – typically a directory on ASCII or a PDSE or zFS directory on zOS
ODSFILE= the name of the output that will be created
ANALID - Example
%ANALID( READSMF=YES, PDBOUT=PDB, PRINT=YES, ODSTYPE=HTML, ODSPATH=E:\. ODSFILE=ANALID.HTML);
ANALID – Sample
ANALID - Sample
ANALID - Sample
ANALID – Sample
ANALID - Sample
ANALID - Sample
ANALID - Sample
ANALID - Sample
ANALID - Sample
ANALGRID
Creates a dense color coded grid of values using PROC REPORT
Does not require SAS/GRAPH Works on all SAS versions 9.1.3 and
above
ANALGRID
Example 1– Read ASUM70LP and for the specified
system create a grid of CPU busy for a day.
– This is the default with addition of an INCODE to select a specific LPAR
%ANALGRID(INCODE=IF LPARNAME=SYSG;);
ANALGRID
ANALGRID
Example 2 – compare year to year same month excluding weekdays and holidays
– %ANALGRID(– INDATA=RMFINTRV,– SORTBY=SYSTEM MONTH,– SYSTEM=SYSG,– INCODE=MONTH=DATEPART(STARTIME)-DAY(DATEPART(STARTIME))+1;– FORMAT MONTH MONYY.;– if 1 lt weekday(datepart(startime)) lt 7;– if month(datepart(startime))=1;– if datepart(startime) not in('26dec11'd,'24nov11'd,'25nov11'd,– '05sep11'd,'04jul11'd,'30may11'd,'21feb11'd,'17jan11'd,'24dec10'd,– '25nov10'd,'26nov10'd,'16jan12'd,'02jan12'd,'16jan12'd,'20feb12'd);,– TITLE1=% CPU Busy,– VARIABLE=pctcpuby,VARLABEL=% CPU,varformat=5.2,– ROWVARIABLE=DATE,ROWLABEL=DATE,ROWFORMAT=DATE.,– ODSPATH=e:,ODSFILE=april.html);
ANALGRID
ANALGRID
ANALGRID
You have complete control of– Colors and levels– Column and row variables– Column and row labels– Column and row formats
ANALGRID
%ANALGRID( SYSTEM=SYSG, INDATA=RMFINTRV, SORTBY=SYSTEM, VARFORMAT=TIME12.2, dates=lastweek, BKT1='01:00'T/BLUE/WHITE, BKT2='02:00'T/GREEN/WHITE, BKT3='03:00'T/CYAN/BLACK, BKT4=, WEIGHT=, SORTLABEL=System, STAT=SUM, VARIABLE=CPUTM, odspath=e:, odsfile=cputime.html, VARLABEL=CPU TIME, COLVARIABLE=TIME,COLLABEL=TIME,COLFORMAT=TIME5.,
ROWVARIABLE=DATE,ROWLABEL=DATE,ROWFORMAT=DATE.
);
ANALGRID
VMXGPRNT
Utility to print any SAS dataset with labels modified to include the variable name and/or create a comma delimited output (CSV).
VMXGPRNT – Parameters
SP_DSET – dataset to be printed – defaults to _LAST_
SP_NOBS – number of OBS to be printed – defaults to 20
SP_REMV – remove * from labels in CSV file – defaults to NO
VMXGPRNT – Parameters
TMPPRNT – destination for a temporary dataset – on zOS it will be constructed and dynalloc’ed as a temporary dataset but on ASCII will be placed in your SASUSER directory. Defaults to TMPPRNT.SAS
BYLST – list of BY variables – defaults to a null string
VMXGPRNT – Parameters
VARLST – list of variables to be printed. Default is a null string which will print all variables
NOEXIMSG – suppresses various warnings/notes – default is YES
SP_OPNS – PROC PRINT options default is SPLIT=‘*’
VMXGPRNT – Example 1
%VMXGPRNT(SP_DSET=PDB.DB2ACCT,SP_NOBS=3);
Print PDB.DB2ACCT
VMXGPRNT – Example 1
VMXGPRNT – Example 2
Create a CSV file– Filename csv ‘h:\mxg\vmxgprnt.csv’;– ods csvall file=csv;
– %vmxgprnt(SP_DSET=PDB.DB2ACCT,SP_NOBS=3,sp_remv=Y);
– run;
– ods csvall close;
– run;
VMXGPRNT – Example 2
VMXGFIND
Utility that will find every OBS in every dataset where some condition is satisfied and make a copy/print the observations.
For example:– Find all obs where JOB=:’CICS’
VMXGFIND – Parameters
PDB= LIBNAME to be searched – default is PDB – can be 1 or many
PDBOUT= where to put the output datasets – datasets here will be named DDNAME_dataset where DDNAME is the libname where they were found
VMXGFIND – Parameters
KEEPIN= a list of variables that are used in the comparison
FIND= the comparison – for example…– Job=:’CICS’
– KEEPIN=STARTIME STRTTIME INTBTIME,– FIND= IF ('31JAN2010:10:11:12'DT LE STARTIME LE '31JAN2010:22:23:24'DT )
OR ('31JAN2010:10:11:12'DT LE STRTTIME LE '31JAN2010:22:23:24'DT ) – OR ('31JAN2010:10:11:12'DT LE INTBTIME LE '31JAN2010:22:23:24'DT ) ;,
VMXGFIND – Parameters
PRINT= default is NO– YES – print all the observations– NO – no print– xxx – print xxx observations
VMXGFIND
If PRINT=YES or xxx then VMXGPRNT is used to do the printing
Example 1:– %VMXGFIND(FIND=QWHSSSID=DBTB,PRINT=3);
VMXGFIND
VMXGSRCH
Utility that will find every observation in every dataset in every allocated SAS data library where the value of the observation contains some string.– Note: libraries must have been allocated
either explicitly (LIBNAME statement) or by a DATA/PROC step.
VMXGSRCH – Parameters
LIBNAME= the libname to be searched. Default is a NULL string. _ALL_ will search all allocated SAS data libraries (they don’t have to be MXG) and anything else will search that specific LIBNAME. Only LIBNAMEs that have been opened will be found!!!!! You may need to insert a LIBNAME on zOS.
VMXGSRCH - Parameters
COPYTO= copy the datasets and observations that match to this LIBNAME
NOBS= the number of OBS to print – default is MAX
LOG= a large number of lines may be generated – LOG=NO suppresses them. Default is YES
VMXGSRCH - Parameters
VALUE – the value to search for Results= what you want us to do
– PRINT – just print the obs/datasets that match– COPYONLY – copy the datasets but don’t print– COUNT – just produce a count of datasets/obs/variables
that match– LABEL – produce a list of variables/datasets where the
value is in the label– FORMAT – produce a list of variables/datasets where the
value is in the format
VMXGSRCH – Example 1
%VMXGSRCH( LOG=NO,RESULTS=COUNT, VALUE=D2DD,LIBNAME=PDB);
VMXGSRCH- Example 1
VMXGSRCH – Example 2
%VMXGSRCH( LOG=NO,RESULTS=PRINT,NOBS=2, VALUE=D2DD,LIBNAME=PDB);
VMXGSRCH – Example 2
VMXGSRCH – Example 3
%VMXGSRCH( LOG=NO,RESULTS=PRINT,NOBS=2, VALUE=D2DD,LIBNAME=PDB, COPYTO=WORK);
VMXGSRCH – Example 3
VMXGSRCH – Example 4
%VMXGSRCH( LOG=NO,RESULTS=COPYONLY, VALUE=D2DD,LIBNAME=PDB, COPYTO=WORK);
VMXGSRCH – Example 4
VMXGSRCH – Example 5
%VMXGSRCH(VALUE=CPU,RESULTS=LABEL);
NOTE: Values are case sensitive
VMXGSRCH – Example 5
VMXGSRCH – Example 6
VMXGSRCH(VALUE=TIME,RESULTS=FORMAT);
VMXGSRCH – Example 6
UTILWORK
Don’t understand the documentation on defining your workloads to RMFINTRV? This utility will build you a skeleton RMFINTRV member based on your TYPE72GO records.
UTILWORK - Parameters
PDB= may be either SMF or some libname that contains a TYPE72GO dataset. SMF is preferred since the normal _ETY72GO exit will suppress service classes with no activity in an interval. You only need to use a single RMF interval.
UTILWORK – Parameters
USEREPRT= YES/NO do you want to use report classes or service classes to define workloads. Strongly recommended that you use report classes since there can be many many more at no real cost.
UTILWORK - Example
%UTILWORK(PDB=PDB, OUTFILE=RMFINTRV, USERPRT=YES, INTERVAL=QTRHOUR)
UTILWORK - Example
READDB2
MXG supplied macro that generates the code to read all of the different types of DB2 SMF data (all IFCIDs). It has been ‘enhanced’ to make a copy of the SMF data and allow for selection based on reading the record headers only which makes it very fast.
READDB2
For a full list of parameters and usage see READDB2 member in the MXG SOURCLIB
Concentration here will be on selection parameters and copying of SMF data
READDB2
SMFOUT= DDNAME to which SMF data will be copied – if blank no copy is made
COPYONLY= YES/NO – only copy SMF data do not format SAS datasets– Useful to make mini-SMF files to feed to
DB2PM or send off to vendors PDBOUT= DDNAME to which SAS datasets
are written (WORK is default if left blank)
READDB2 - Parameters
SYSTEM – list of systems PLAN – list of plan names AUTHID – list of authorization IDs CORRID – list of correlation IDs CONNID – list of connection IDs DB2 – list of DB2 subsystems CONNTYPE – list of connect types
READDB2 - Parameters
TRANNAME – list of end-user transaction names
PACKAGE – list of package names SMFBEGIN =SAS datetime constant –
starting point of data SMFEND – SAS datetime constant –
end point of data– SAS datetime constants are of the form
01sep10:01:30:00 – no quotes are needed
READDB2
All values in lists separated by spaces All parameters separated by commas (except the last
one) All values are automatically wild carded – that is,
however many bytes are in the value is the length of the compare
SMFBEGN= earliest time in form ddmmmyy:hh:mm:ss or 10OCT08:15:00:00
SMFEND= latest time in same form
READDB2
%READDB2(TRANNAME=OLB_DISP, COPYONLY=YES,SMFOUT=SMFOUT);– Copy records where TRANNAME starts with
OLB_DISP to SMFOUT DD but do not create SAS datasets
%READDB2(TRANNAME=OLB,PDB=WORK, SMFOUT=SMFOUT);– Copy records where TRANNAME starts with OLB
and also place them in SAS datasets in the WORK dataset
VMXGSUM
Generalized summarization of ANY SAS dataset
– Uses PROC MEANS to do summarization– SORTs data– Allows for changes in input and output data– Optimizes variables kept – Carries labels and formats thru summarization– Allows for long variable names– Allows for normalization of variables and changing time intervals
VMXGSUM
Common in reporting:– DATA xxxx;– SET yyyy;– PROC SORT DATA=xxxx;– PROC MEANS DATA=XXXX OUT=zzzz;– DATA final;– SET zzzz;
VMXGSUM
VMXGSUM is a short-hand way of coding a repetitive set of commands.
Used extensively internally in many MXG members but especially common in ASUM**** and TRND**** members.
VMXGSUM - SYNTAX
%VMXGSUM(– INDATA= input dataset(s) name– OUTDATA= output dataset name– SUMBY= list of variables by which data should
be sorted– INCODE= a stub of SAS code executed during
the first data step– OUTCODE= a stub of SAS code executed
during the final data step
VMXGSUM - SYNTAX
– INTERVAL= how to change the time interval. Valid values are:
QTRHOUR HALFHOUR HOUR THREEHR MINUTE WEEK MONTH MYTIME
– DATETIME= the variable name of the variable containing the datetime value on which INTERVAL= will be applied
– SYNC59= if your time is synched to 59 minutes, will add 60 seconds before calculating interval if set to YES
VMXGSUM - SYNTAX
ID= list of variables that will be carried forward as ID values
AUTONAME=YES/NO AUTONAME = YES says to use the autonaming functions of SAS V8 to name the output variables.
– This allows the specification of the same variable name in multiple lists but changes the output variable name to variable_suffix where suffix is the name of the function performed on the variable.
VMXGSUM - SYNTAX
SUM= list of variables to be summed MAX= list of variables to be maxxed MIN= list of variables to be minned MEAN= list of variables to be meaned P1= list of variables to get percentile 1 P5= 5th percentile variables P10= 10th percentile variables
VMXGSUM - SYNTAX
– P25 P50 P75 P90 P95 P99 - percentile values
– STD - Standard Deviation– VAR - variance– CV - coefficient of variance– STDERR - Standard error– KURTOSIS - Kurtosis– T - T value
VMXGSUM - Syntax
NORM1-NORM99 - normalization of data. Maintaining rates as rates and not averages of averages. On the front-end, the rate has to be multiplied by the duration and on the back end divided again to recalculate the correct rate.
VMXGSUM - SYNTAX
– NORM1-NORM99 - syntax rate1 rate2 rate3…ratex/duration
List the variables to be normalized followed by a / then the variable to be used to do the normalization.
VMXGSUM - SYNTAX
There are other parameters. See the documentation in the member for usage and the member ADOCSUM.
VMXGSUM - Example 1
Summarize the dataset TYPETMNT by DEVICE and TMNTTIME calculating average mount delay and the total number of mounts per quarter hour.
%vmxgsum( indata=pdb.typetmnt, outdata=tapemnts, sumby=device tmnttime, interval=qtrhour, datetime=tmnttime, mean=tapmnttm, freq=mounts);
VMXGSUM - Example 2
Summarize the Goal Mode type 72 records for the TSO service class calculating the average response time, the number of transactions at one hour intervals by period.
VMXGSUM - Example 2
%VMXGSUM( INDATA=PDB.TYPE72GO, OUTDATA=TSOSUM, SUMBY=STARTIME PERIOD, INCODE= IF SRVCLASS=‘TSO’;, SUM=RESPAVG NUMTRAN, NORM1=RESPAVG/NUMTRAN, INTERVAL=HOUR, DATETIME=STARTIME);
VMXGSUM Usage Notes
NORMx operands must be contiguous starting at 1. That is, you cannot have NORM1 and NORM3 without a NORM2.
VMXGSUM Usage Notes
The first data step is almost always converted to a VIEW rather than a real data step.
KEEPALL=NO is resource intensive and not really needed except in odd cases. KEEPALL=YES is much preferred. The keep lists on all output datasets are optimized regardless of KEEPALL setting.
Why VMXGSUM?
So why not just use PROC MEANS with CLASS operands?
VMXGSUM in tests is usually much more efficient and in some cases will do the summarization where using PROC MEANS or PROC SUMMARY with CLASS operands runs out of memory.
This is especially true with the current release of SAS (9.1.3 SP4) on zOS which is defaulting to using THREADS.
ANALCNCR
Counts concurrent events. How many of something were happening at the same time.
ANALCNCR - History
Method used in original release of MXG:– DO TIME=BEGIN TO END BY 5;– OUTPUT;– END;– Then add up all the observations with a
given value of TIME. Created a HUGE number of observations and was cumbersome.
ANALCNCR - History
Method used with ANALCNCR:– TIME=BEGIN;COUNT=1;OUTPUT;– TIME=END;COUNT=-1;OUTPUT;
– Now add up the counts by time and you are done (basically.) Many many fewer observations.
ANALCNCR - History
If there are three tape allocations:
– Allocation 1 begins at 08:00 ends at 08:30– Allocation 2 begins at 08:15 ends at 08:25– Allocation 3 begins at 08:20 ends at 08:45
ANALCNCR - History MAX of 3 concurrent allocations
– 15 minutes of 1 – 5 minutes of 2 – 5 minutes of 3 – 5 minutes of 2 – 15 minutes of 1
Old method– Allocation 1 - 1800/5=360 obs– Allocation 2 - 600/5=120 obs– Allocation 3 - 1500/5=300 obs– Total = 780 obs
New Method– Each allocation is 2 OBS– Total = 6
ANALCNCR - Example 1
How many jobs are running concurrently in class A average and max.
%ANALCNCR(INDATA=PDB.JOBS, OUTSUMRY=RUNTIME, SUMBY=JOBCLASS, INCODE=IF TYPETASK=‘JOB’;, INTERVAL=QTRHOUR, STARTIME=JINITIME, ENDTIME=JTRMTIME, OTCODESM= AVGRUN=CONCURNT/DURATM; RENAME MAXCNCR=MAXRUN;);PROC PRINT;ID JOBCLASS TIMESTMP;VAR AVGRUN MAXRUN;
ANALCNCR - Example 2
Now suppose you want the INPUT QUEUE time for the same job class.
%ANALCNCR(INDATA=PDB.JOBS, OUTSUMRY=QUETIME, SUMBY=JOBCLASS, INCODE=IF TYPETASK=:’JOB’;, INTERVAL=QTRHOUR, STARTIME=READTIME, ENDTIME=JINITIME, OTCODESM= AVGQUE=CONCURNT/DURATM; RENAME MAXQUE=MAXRUN;);PROC PRINT;ID JOBCLASS TIMESTMP;VAR AVGQUE MAXQUE;
ANALCNCR - Example 3
Now put the two outputs together
DATA JOBSTAT;MERGE RUNTIME QUETIME;BY JOBCLASS TIMESTMP;PROC PRINT;ID JOBCLASS TIMESTMP;VAR AVGQUE AVGRUN MAXQUE MAXRUN;
ANALCAPD Can you save money by capping the MSU’s consumed? Billing is based on the peak of the rolling 4 hour MSU average Rolling average will (almost) always lag behind actual usage So, you can set a cap lower than the actual peak and possibly
reduce software billing ANALCAPD will let you ‘play’ with values to find a happy MSU
value that allows work to run while reducing the peak MSU value
ANALCAPD
Uses the ASUMCEC dataset in the PDB as input
Best granularity is when you match CECINTRV to INTERVAL in ASUM70PR
ANALCAPD – Parameters
PDB=PDB – where is the ASUMCEC data GRAPHICS=YES – use SAS/GRAPH (it will
detect if it is not there) DEFCAP= the MSU value you want to ‘model’ CECINTRV=HOUR – the CECINTRV value in
use – QTRHOUR HALFHOUR etc
ANALCAPD - Results
ANALCAPD – Results
Black line is current capacity Cyan line is current cap (in this case
there is not one) Blue line is actual usage Green line is rolling 4 hour average Red * are the intervals where the CEC
would have been capped