Building an MXG PDB from NTSMF Data
• ‘Classic’ MXG PDB Structure
• Why it doesn’t work
• What needed to change
• How it is implemented for NTSMF
• Using BLDNTPDB
Classic MXG PDB Structure
• Read ALL the data and place it in a PDB with possibly CICSTRAN on the side
• Run daily - daily contains detail of ALL data read
• WEEKBLD consolidates dailies into weeklies
• MONTHBLD consolidates weeklies into monthlies
Classic MXG PDB Structure
• DASD storage required:– 8 daily PDB– 5 weekly PDB– enough WORK space for the largest
dataset (STEPS or TYPE74)
Why It Doesn’t Work
• 40 GB of SMF data daily
• Needs 160GB of WORK space (SORTWK and WORK)
• 320GB of space for daily datasets
• 1+ Terror Bytes for weekly datasets
• Time - Daily jobs that run for more than a day are not practical
What Needed to Change
• Multiple Parallel Threads
• Run week-to-date
• Run month-to-date
• Eliminate weekly/monthly jobs
• Exploit summarization macros where needed to reduce volume
How Is It Implemented for NTSMF?
• BLDNTPDB - a very large and complex collection of SAS MACROs that allows the user to choose how the PDB should be structured.
• Can use the classic MXG structure or...
• Can use virtually any structure the user chooses
Using BLDNTPDB
• Parameters and Meanings
• Examples of use
• Future Directions
• ?????????????????????
Using BLDNTPDB - Parameters and Meanings
• NTSMF=C:\MXG\NTSMF\NTSMF.SMF - where is your RAW NTSMF data?
• WEEKSTRT=MON - on what day of the week does your week start? The normal MXG default is Monday. This controls when weekly processing takes place.
• RUNNTINT=YES - should we run the NTINTRV summarization?
Parameters and Meanings
• ASUMDUR=HOUR - what level of summarization should we apply? Valid values are QTRHOUR, HALFHOUR, HOUR, SHIFT, DATE, WEEK, MONTH
• KEEPERS=null - A list of datasets to be ‘kept’ (usually not used)
• DROPPERS=null - A list of datasets to be ‘dropped’ (built but not kept in the output PDB.)
Parameters and Meanings
• ZEROOBS=null - Similar to DROPPERS but datasets are created with 0 observations
• MAXDRVS=26 - Maximum number of logical drives/physical drive (limits and array size.)
• RERUN=null - If YES is specified forces the rewriting of an old PDB USE WITH CAUTION
Parameters and Meanings
• MON=C:\MXG\MON• TUE=C:\MXG\TUE• WED=C:\MXG\WED• THU=C:\MXG\THU• FRI=C:\MXG\FRI• SAT=C:\MXG\SAT• SUN=C:\MXG\SUN• Dataset placement for DAILY PDBs. If a null string is
used, the dataset WILL NOT BE BUILT.
Parameters and Meanings
• WEEK=C:\MXG\WEEK
• WEEK2=C:\MXG\WEEK2
• WEEK3=C:\MXG\WEEK3
• WEEK4=C:\MXG\WEEK4
• WEEK5=C:\MXG\WEEK5
• Placement for weekly datasets. If a null string is used, the dataset WILL NOT BE BUILT.
Parameters and Meanings
• TREND=C:\MXG\TREND
• MONTH=C:\MXG\MONTH
• MONTH1=null
• Placement for monthly and TREND datasets. A null string results in the dataset NOT BEING BUILT.
Parameters and Meanings
• RUNDAY=YES - Run the BUILD process daily and build the specified daily databases
• RUNWEEK=NO - YES means run the weekly BUILD process if it is the day specified by WEEKDATE and WTD means build WTD datasets daily
• RUNMONTH=NO - YES means run the monthly BUILD process on the 2nd day of the month and MTD means build MTD datasets daily
• RUNTREND=WEEKLY - Build the TREND datasets when the WEEKLY BUILD is run. If DAILY is specified, build the TREND datasets when the DAILY datasets are built.
Parameters and Meanings
• RPTRAW=NO - Build a table of the distribution of the RAW NTSMF record types
• RPTDAY=PDB - Create reports for the most recent PDB at the hourly
• RPTWEEK=NO - Create reports for the WEEKLY PDB if it has just been
• RPTMONTH=NO - Create reports for the MONTHLY database if it has just been created
• RPTTREND=NO - Create reports for the TREND database if it has just been updated
Parameters and Meanings
• GRAFDAY=PDB - Create reports for the most recent PDB at the hourly
• GRAFWEEK=NO - Create reports for the WEEKLY PDB if it has just been
• GRAFMNTH=NO - Create reports for the MONTHLY database if it has just been created
• GRAFTRND=NO - Create reports for the TREND database if it has just been updated
Parameters and Meanings
• Any combination of RUNxxxx, RPTxxxx, and GRAFxxxx specifications may be used. You can do DAILY, WEEKLY, and MONTHLY runs in a single execution as well as creating reports and graphs at each level.
Parameters and Meanings
• Null strings passed as some parameters means ‘dont do this!’
Examples of Use
• Classic MXG Structure
• PDB for each day of week, 5 weeks, monthly with week run on Monday– %BLDNTPDB(RUNDAY=YES,
RUNWEEK=YES, RUNMNTH=YES, RUNTRND=WEEKLY);
Examples of Use
• Single daily PDB no WEEKLY/MONTHLY/TREND– %BLDNTPDB(RUNDAY=YES,MON=,
TUE=, WED=, THU=, FRI=, SAT=, SUN=);
Examples of Use
• Single daily PDB with week-to-date, month-to-date, and trending done daily– %BLDNTPDB(RUNDAY=YES,
RUNWEEK=WTD,RUNMNTH=MTD, RUNTRND=DAILY,MON=,TUE=,WED=, THU=, FRI=, SAT=, SUN=);
A Few Words About MXG 16.04 and Later
• Each dataset now has an up to 6 character mnemonic identifier. In the case of NTSMF data it starts with NT and then has 4 unique characters.
• This is used to identify what datasets to KEEP DROP or set to ZEROOBS.
• YOU MUST REVIEW THE 16.04 CHANGES and DOCUMENTATION!
Examples of Use
• Build and keep only the datasets needed for NTINTRV– %BLDNTPDB(KEEPERS=NTBROW
NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);
Examples of Use
• Build only the datasets needed for NTINTRV and DROP after NTINTRV is created: – %BLDNTPDB(DROPPERS=NTBROW
NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROC NTPROR NTRDIR NTSERV NTSYST NTTCP);
Examples of Use
• Build the datasets needed for NTINTRV but build the PROCESS dataset with zero observations:– %BLDNTPDB(KEEPERS=NTBROW
NTCACH NTFTP NTMEM NTNBTC NTNBUI NTNETS NTNWLI NTNWLN NTNWLS NTOBJ NTPAGE NTPDSK NTPROR NTRDIR NTSERV NTSYST NTTCP, ZEROOBS=NTPROC);
Examples of Use
• Reporting can be added to any execution:– %BLDNTPDB(xxxxxxxxxxxxxxx,
RPTDAY=YES);
Examples of Use
• Reporting can be done against any PDB:– %BLDNTPDB(xxxxxxxxxxxxxxx,
RPTDAY=MON);
Examples of Use
• GRAPHS can be created for daily, weekly and trend datasets (CPU usage only at this time - more to come.)– %BLDNTPDB(xxxxxxxxxxxxx,
GRAFDAY=YES, GRAFWEEK=YES);
Examples of Use - A Case Study
Suppose you want the following:Run the PDB DAILY keeping all datasetsMaximum of 5 logical drives/physical driveStart the week on SundayKeep a WTD/MTD/and TREND datasetsRun reports of Daily usageCreate Graphs of Daily usage
Examples of Use- A Case Study
%BLDNTPDB(MON=,TUE=,WED=,THU=,FRI=,SAT=,SUN=,WEEK2=,WEEK3=,WEEK4=,WEEK5=,RUNDAY=YES,WEEKSTRT=SUN,RUNWEEK=WTD,RUNMNTH=MTD,RUNTRND=DAILY,MAXDRVS=5,RPTDAY=PDB,GRAFDAY=PDB);
Future Directions
• Largely up to YOU!
• My knowledge of the data is limited
• My experience with the data is limited
• I need to know what YOU want/need
Future Directions
• What reports are needed– Data sources– Data elements– Levels of summarization
Future Directions
• What graphs are needed– What data sources– What kinds of graphs– What levels of summarization
Future Directions
• Currently on the development list:– Selection of datasets to place in the
WEEKLY/MONTHLY databases - a WEEKKEEP= operand or something along those lines
– More automated summarization (this is a tough one because it supposes knowledge of the datasets and variables that is difficult to have programmatically.)
Future Directions
• Currently on the Development List– Ability to limit variables in WTD/MTD
datasets using DROP=– Ability to output reports as HTML
???????????