IBM Software Group

®

WebSphere® Support Technical Exchange

Using File Nodes with WebSphere Message Broker 6.1

Thomas BienSoftware Engineer

IBM Software Group

WebSphere® Support Technical Exchange 2 of 23

Agenda Previous WMB File Capabilities

File Node Basics

Flow Demonstration

Other Node Properties

Advanced Options

FTP Possibilities

Troubleshooting and Recovery

IBM Software Group

WebSphere® Support Technical Exchange 3 of 23

Product Extensions:- Message Broker File Extender (MBFE)

SupportPacs:- IA11 Queue Sequential Access Method (QSAM) on z/OS- IA13 Virtual Storage Access Method (VSAM) on z/OS

Built in nodes:- Trace Node (meant for debugging)

Previous WMB File Capabilities

IBM Software Group

WebSphere® Support Technical Exchange 4 of 23

File Node Basics: The FileInput Node

The FileInput Node:

Basic Properties:- Input Directory?- File name or pattern?- Action on successful processing?

- Move to Archive Subdirectory (mqsiarchive)- Add Time Stamp and Move to Archive Subdirectory- Delete*

- Replace duplicate archive files?

IBM Software Group

WebSphere® Support Technical Exchange 5 of 23

File Node Basics: The FileOutput Node

The FileOutput Node :

Basic Properties:

- Directory?- File name or pattern?- Output file action?

- Replace Existing File* (or Create if File does not Exist)- Create File (Fail if File Exists)- Archive and Replace Existing File (mqsiarchive)- Time Stamp, Archive and Replace Existing File

- Replace duplicate archive files?

IBM Software Group

WebSphere® Support Technical Exchange 6 of 23

File Node Basics: The LocalEnvironment Tree The LocalEnvironment tree is used to maintain file node settings

across a message flow.

The FileInput node will populate the ‘File’ and ‘Wildcard’ sub-trees for each message it sends through its out terminal.

The FileOutput node will populate the ‘Written Destination’ tree.

IBM Software Group

WebSphere® Support Technical Exchange 7 of 23

Flow Demonstration: Simple Flow:

Input Directory: C:\wste_sample\input Input File Pattern: *.in

Output Directory: C:\wste_sample\output Output File Pattern: *.out

Trace Directory: C:\wste_sample\trace

IBM Software Group

WebSphere® Support Technical Exchange 8 of 23

Flow Demonstration: Under the Covers Simple Flow:

1) FileInput: checks ‘C:\wste_sample\input’ for ‘*.in’ and finds ‘test.in’2) FileInput: stores file info to LocalEnvironment (with ‘test’ as a wildcard)3) FileInput: creates ‘C:\wste_sample\input\mqsibackout’ (in case of errors)4) FileInput: sends output message through the out terminal (removes ‘test.in’)

1) Trace1: writes ‘trace1.out’ to ‘C:\wste_sample\trace’2) Trace1: sends output message through the out terminal

1) FileOutput: writes temp messages to ‘C:\wste_sample\output\mqsitransit’2) FileOutput: transfers ‘*.out’ to ‘C:\wste_sample\output’ (pulling from wildcard)3) FileOutput: sends output message through the out terminal

IBM Software Group

WebSphere® Support Technical Exchange 9 of 23

Other Node Properties: Common Properties “Input Message Parsing” contains the

same domain and parser settings asfound in other WMB Input nodes.

“Validation” settings are found onall WMB input and output nodes.

“Instances” controls settings foradditional instances of the node,the same as in the MQInput node.

IBM Software Group

WebSphere® Support Technical Exchange 10 of 23

Other Node Properties: Polling The Polling interval controls how often the FileInput node

accesses the input directory to check for files that matchthe file name or pattern.

The default setting is to check every 5 seconds.Pros for decreasing this value:

•Increased availability•Shorter response times

Cons for decreasing this value:•Increased flow processing and CPU time

The inverse is true when you increase the polling interval

IBM Software Group

WebSphere® Support Technical Exchange 11 of 23

Other Node Properties: Transactions While file system interactions are non-transactional,

the rest of the flow can be placed under ‘syncpoint’ control.

IBM Software Group

WebSphere® Support Technical Exchange 12 of 23

Other Node Properties: FileOutput Request FileOutput “Request” settings can make use of the message body

as well as the LocalEnvironment tree to allow dynamically defined output settings.

Request Overrides: - Data location (‘$Body’) - Request directory property location

(Destination/File/Directory) - Request file name property location

(Destination/File/Name)

IBM Software Group

WebSphere® Support Technical Exchange 13 of 23

Advanced Options: FileInput Records An input file is not the same as an input message. An input file can contain one or more “records”. Each record corresponds to a single input message.

How do we distinguish the number of records in an input file?“Records” tab allows us to specify the detection rules:

•Whole File•Fixed Length•Delimited•Parsed Record Sequence

The FileInput node automatically sends a message through the “End of Data” terminal when it finishes reading a file.

IBM Software Group

WebSphere® Support Technical Exchange 14 of 23

Advanced Options: FileOutput Records The FileOutput Records tab, similarly, determines the number of

flow messages that are written to a single output file.

The ‘Finish File’ input terminalcontrols when to finalize the file.

IBM Software Group

WebSphere® Support Technical Exchange 15 of 23

FTP Possibilities: Through WMB 6.1.0.3, both file nodes contain an FTP tab with

a checkbox to set whether FTP settings should be used.

If set, the FTP settings are used to transfer files to the remote server, while all processing is done on local copies of the files.

IBM Software Group

WebSphere® Support Technical Exchange 16 of 23

FTP Possibilities: The “Server directory” points to the directory on the FTP server

you wish to scan for your input files. It should be formatted for the file system standards of the FTP Server, not the local machine.If unset, the default is to use the server’s default login directory.

Transfer mode sets whether the file should be transferred to the local system as binary or ASCII.

FileInput uses ‘ScanDelay’ to determine how often to poll the remote file system (instead of the ‘polling’ settings).

FileOutput uses ‘Retain local file after transfer’ to allow the user to keep a copy of the output file locally, if needed.

IBM Software Group

WebSphere® Support Technical Exchange 17 of 23

FTP Possibilities: When specifying FTP settings, you can configure them on

the tab, or through a configurable service:mqsicreateconfigurableservice myBrokerName -c FtpServer

-o myServiceName -n <propertyName> -v <value>

Valid FtpServer configurable service properties:serverName, accountInfo, scanDelay, transferModeconnectionType, remoteDirectory, securityIdentity

The security identity in use is created with the command:mqsisetdbparms -n ftp::myIdentity -u myUserName -p myPassword

When running mqsisetdbparms, include the “ftp::” prefix to your identity name so it can be used with file node FTP settings.

IBM Software Group

WebSphere® Support Technical Exchange 18 of 23

FTP Possibilities: SFTP As of 6.1.0.3, the file nodes do not have the ability to use

Secure FTP (SFTP) servers.

APAR IC60152 has been opened to address this:http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg1IC60152

The fix is targeted for fixpack 6.1.0.4, which is scheduled for release in 2Q 2009:

http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg27006308

IBM Software Group

WebSphere® Support Technical Exchange 19 of 23

Troubleshooting and Recovery: All errors are output through normal WMB error outputs:

(UNIX® syslog; Windows® application event logs, z/OS JOBLOG)

When a flow starting from a file fails, retry settings dictate the FileInput Node’s response:

“failure”: message moves to failure tab (or backed out)“short retry”: immediate retry(s) of the message through the flow“short and long retry”: multiple sets of “short retry” attempts

If a file fails, even after retry, the file can either be backed out to a subdirectory (mqsibackout), or deleted.

IBM Software Group

WebSphere® Support Technical Exchange 20 of 23

Summary

Message Broker 6.1 contains new, built-in nodes specifically for file I/O.

File nodes can be customized to interact with files in a variety of formats.

Files can be accessed locally, or through FTP.

IBM Software Group

WebSphere® Support Technical Exchange 21 of 23

Additional WebSphere Product Resources Discover the latest trends in WebSphere Technology and implementation, participate in

technically-focused briefings, webcasts and podcasts at: http://www.ibm.com/developerworks/websphere/community/

Learn about other upcoming webcasts, conferences and events: http://www.ibm.com/software/websphere/events_1.html

Join the Global WebSphere User Group Community: http://www.websphere.org

Access key product show-me demos and tutorials by visiting IBM Education Assistant: http://www.ibm.com/software/info/education/assistant

View a Flash replay with step-by-step instructions for using the Electronic Service Request (ESR) tool for submitting problems electronically: http://www.ibm.com/software/websphere/support/d2w.html

Sign up to receive weekly technical My Notifications emails: http://www.ibm.com/software/support/einfo.html

IBM Software Group

WebSphere® Support Technical Exchange 22 of 23

IBM Support Wants to Hear From You!

1. Visit any product support pages on IBM.com.2. Click on “Participate in Questionnaire” on top right of page.3. Takes 5-10 minutes to complete.

Tell us about your support needs and wants

Or go to https://www.ibm.com/survey/oid/wsb.dll/s/ag21f?wsb34=swg_user

IBM Software Group

WebSphere® Support Technical Exchange 23 of 23

Questions and Answers