wmb topten problems

IBM Software Group

®

Michael S. Patton

[email protected]

WebSphere® Support Technical Exchange

WebSphere Message Broker Top Ten Problems

A Level 2 Perspective

IBM Software Group

WebSphere® Support Technical Exchange 2

Agenda1) Failing Deploys

2) Toolkit is slow or terminates all together3) DFE Memory Leak

4) Debugger (RAC) not working (Remote Agent Controller)

5) Broker and RAD (Rational Application Developer)

6) Message Flow Hangs

7) Publish and Subscribe8) MRM and Validation

9) eSupport

10) Noteworthy APARS and Useful Websites

IBM Software Group


Failing Deploys

� Three possible scenarios

�No response at all

�Negative response

�Error on deploy

� Common Deploy Error messages

�BIP2080, BIP2066, BIP2087, & BIP1536

(See additional notes in the Addendum)

IBM Software Group


Failing Deploy – No Response

� Check Event Viewer / message logs

�Other messages give a clue as to what happened.

� Check for MQ FDC errors

�MQ Series could not send the deploy message for some reason

� No responses should still result in a timeout response

IBM Software Group


Failing Deploy – No Response

� Infrastructure problems� Large deploy message?

�MAXMSGLENGTH large enough?

� MQ Objects

�Any errors with MQ Series

�Check MQ logs.

� MQ Return codes propagated through BIP2070.� Database error codes propagated through BIP2322


IBM Software Group


Failing Deploy – Negative Response� BIP1536E: Configuration Manager unable to register for

internal subscriptions on the broker

�The deploy was processed

�The configmgr simply cannot receive updates from broker

�Thus Toolkit will not reflect the current status of the broker

�BIP7150E: User id not authorized to put to queue on queue manager

�The configmgr service id must be defined on broker box and must have permission to put to transmit queue on broker queue manager to send messages to theconfigmgr QM

IBM Software Group


Failing Deploy – Negative Response

� BIP2080E, BIP2066E� Configuration timeout and ConfigurationDelayTimeout

�Are they set high enough?

�Mqsichangebroker <brokername> -g <value> -k <value>

� -g = how much time to allow for the deploy

� -k = how much time to wait before starting the –g timer.

� Complex Flows tale time to quiesce� Deploy to EG with large number of flows takes time� Messages currently in mid-stream will either finish or rollback

prior to processing a a new deploy

IBM Software Group


Failing Deploy – Hints and Tips

� Minimize the scope of the deploy

�Deploy smaller groups of message flows and/or sets

�Thus increased deploy performance

� Tune configuration timeout and configuration delay timeout� Remove the flow from the broker and deploy a fresh copy� Back out any updates that were applied since the last deploy� Stop the broker, clear all SYSTEM.BROKER.*.* queues,

restart the broker and deploy again.

IBM Software Group


Failing Deploy – Hints and Tips

� Set the environment variable MQSI_SHOW_DEPLOY

�Works on Windows and Unix

� Inserts a BIP8099I into the message log

�Broker restart needed for variable to be picked up

�Helps to isolate the problem

� Check the message logs for any other MQ, DB or OS errors that may provide a clue

� Questions on Deploys?

IBM Software Group


Toolkit is slow or terminates� Common Problems

�Saving a project is slow

�Create a bar file is slow

�Toolkit is gorging on CPU

�JVM Exceptions

�Debugging

� Saving a project or Create a bar file or Toolkit gorging on CPU

�Message Flow and ESQL validation is memory intensive

�Disable this validation in preferences

�Window ⇒ Preferences ⇒ ESQL and Mapping ⇒ Validation

� Toolkit shuts down due to JVM exception

� Inspect the <install dir>/workspace/.metadata/.log for exceptionmessage

IBM Software Group


Toolkit is slow or terminates

� Debugging Issues

�Large projects and debugging complex flows / testing large messages in Toolkit are resource intensive

�Toolkit (mqsistudio.exe) creates a JVM with default maximum heap size

• The default is ½ the real memory on the system

� Increase this value by starting the toolkit from the command line

�CD to the <Install Dir>\eclipse directory

�Mqsistudio –vmargs –Xmx<value>M

• <value> = number of Megabytes

IBM Software Group


DFE Memory Leak

� DFE is expected to eventually reach a plateau� DFE memory consumption will increase to accommodate the

demands of the flows it contains

�Size and type of message

�Complexity of the flow

� DFE is multithreaded, Flows are threads within� DFE obtains memory from OS and hands it out to threads� DFE cannot return memory to OS

�Threads can return memory to the DFE when a flow ends

�Memory is returned to the OS when the DFE ends

IBM Software Group


DFE Memory Leak� A memory leak may be simply the DFE has not reached a plateau

� IF DFE shuts down due to memory exhaustion, try giving it more memory

�AIX Alter the DataFlowEngine’s executable

� ldedit -b maxdata:0x40000000 DataFlowEngine

• 4 segments = 1 GB

�http://publib.boulder.ibm.com/infocenter/pseries/index.jsp?topic=/com.ibm.aix.doc/cmds/aixcmds3/ldedit.htm

�Env Var: MQSI_THREAD_STACK_SIZE= <value in bytes>

� Increase the JVM Heap size for the broker

�mqsichangeproperties BROKER_NAME -e EG_NAME -oComIbmJVMManager -n jvmMinHeapSize -v 16777216

IBM Software Group


Debugger not working

� Common problems:

�List of Execution Groups do not show up

�List of Message Flows do not show up.

� Agent Controller uses default port 10002 for communication

�Check port status with netstat

�Port must have status of “Listen”

� Is there a firewall?

�Default port on RAC server can be changed

�Edit the file <Rac Install Dir>/config/serviceconfig.xml

�Edit also the toolkit Window ⇒ Preferences ⇒ FlowDebug

IBM Software Group


Debugger not working� Agent controller must be started before the broker� In configuration file “serviceconfig.xml”, change element <Allow

host=“LOCAL”> to <Allow host=“All”>� RAC writes it’s log message to

<RAC_Inst_Dir>/config/servicelog.log� Logging level and detail can be modified in the serviceconfig.xml

file as well

�Default is LoggingLevel=“WARNING” and LoggingDetail=“LOW”

�Change to LoggingLevel=“DEBUG” and LoggingDetail=“HIGH”

� Check to see that installed RAC is correct for CSD


IBM Software Group


Broker and RAD (Rational App Dev)

� WBIMB Version 6� Toolkit V6.0 installs RAD version 6.0.0.1� Toolkit V6.0.0.1 installs RAD version 6.0.1.1� Usually best to let the toolkit install the RAD � Compatibility errors to install Toolkit V6.0 on RAD 6.0.1.1

� Install will fail

IBM Software Group


Message Flow Hangs

� Messages are not picked up from the input queue� Message is stuck in the flow� Exception Handling

�Did you wire a catch or failure terminal?

� Is there a Back Out Queue defined (MQ Series)

�Has the DLQ for the QM been set?

� Aggregate Reply Exception Handling

�Are the unknown, timeout, catch or fail terminals wired?

�Has the MQMD Header been built?

�A message will loop if it cannot be put to a queue.

IBM Software Group


Publish and Subscribe� Collectives (Interbroker communications)

�Communication is TCPIP except for the initial handshake (via MQ)

� If a new broker is added to the collective, or a previous removed broker is re-added, then you must restart all the brokers, and issue a complete topology deploy.

�This forces the brokers to share the initial data for comm via MQ

� Subscription RegOpt “VarUserId”

�Allows any other user to delete/alter the subscription

�Does not indicate multiple owners for a subscription

�Subscriptions are QM:Q:CorrelationId

• How to register multiple topics on the same queue

�APAR IC48412

IBM Software Group


Publish and Subscribe

� Subscription messages sent between cloned brokers are not persistent

�Slight chance that subscription messages shared between cloned brokers will be lost if MQ Series is shut down

�APAR IY74763 will send subscription requests persistent.

� Leading slash (“/”) in Topic will fail

�Using a leading slash in the Topic will cause the EG to continuously respawn

�Also occurs with double slash in the Topic

�APAR IY81324

IBM Software Group


MRM and Validation� Nillable Attributes on XML Elements not removed

�Results in a validation failure

�XML nil=“true” acts as a placeholder for repeats or arrays

�Should simple imply that particular element’s value is NULL

�The schema allows the value to be null, but the attribute nil=“true” will cause a validation error because the element does not have this as an attribute in the model

�Solution: Use NULLXMLSchema in the Physical Format

� Importing a C Header file with a reserved C/C++ keyword

�Using a keyword as a structure name will fail when creating a message definition from a C Header file.

�Cannot use Reserved Keywords as structure or variable names

IBM Software Group


MRM and Validation� XML Parser performance is slow when parsing tag with large

number of entity references.

�The XML Doc contains many entity references ie. <

�Stop the XML parser from parsing the entities

�Enclose in CDATA prior to submitting the msg to the flow

�Process msg as blob to remove the entities

� XMLNS identical Namespaces, different prefixes

�Two NS Prefixes point to the same URL

�Xmlns:NS1=“www.ibm.com” and xmlns:NS2=“www.ibm.com”

�The separate prefixes will not be maintained

�The broker does works with expanded URLs, not prefixes

�The URLs are functionally equivalent

IBM Software Group


Support

� New support initiative geared towards self help� Single interface for known, published problems, PTFs,

APARs and Support Packs� http://www-306.ibm.com/software/integration/

wbimessagebroker/support/� Information from multiple sources is updated daily

IBM Software Group


Support Website

Product

Search

Engine

Download

PTFs and

Support Packs

IBM Software Group


Noteworth APARs and useful Websites

� IC48712 Changing a variable in the toolkit while debugging causes the debugger to disconnect (V6.0)

� JR23282 BOs generated by Ariba Adapter do not match the schema generated by the same framework (All)

� IC48399 Main Map is not using schema of submap (V5.0)� APAR IC48389 Toolkit interpreting different types as the

same type (V5.0)� APAR IC48373 Debug status of Message Flow not preserved

during deploy (V6.0)


IBM Software Group


Noteworth APARs and useful Websites� WMQ Family, WBI Broker’s homepage

� http://www-306.ibm.com/software/integration/wmq/

� http://www-306.ibm.com/software/integration/wbimessagebroker/

� WMQ Support Site

� including Books and Manuals, Lifecycle, Support Packs, Supported Software

� http://www-306.ibm.com/software/integration /wbimessagebroker/support/

� Red Books

� http://www.redbooks.ibm.com

� News Groups

� News://news.software.ibm.com/ibm.software.websphere.mqintegrator/

� http://www.mqseries.net/


IBM Software Group


Additional WebSphere Product Resources

� Discover the latest trends in WebSphere Technology and implementation, participate in technically-focused briefings, webcasts and podcasts at: www.ibm.com/developerworks/websphere/community/

� Learn about other upcoming webcasts, conferences and events: www.ibm.com/software/websphere/events_1.html

� Join the Global WebSphere User Group Community: www.websphere.org � Access key product show-me demos and tutorials by visiting IBM Education

Assistant: ibm.com/software/info/education/assistant

� Learn about the Electronic Service Request (ESR) tool for submitting problems electronically: www.ibm.com/software/support/viewlet/probsub/ESR_Overview_viewlet_swf.html

� Sign up to receive weekly technical My support emails: www.ibm.com/software/support/einfo.html

IBM Software Group


Questions and Answers

IBM Software Group


Addendum

IBM Software Group


Notes (Failing Deploys):

� BIP 2080 : Execution Group <insert_0> failed to return an internal configuration <insert_1> message response within a <insert_2> second time period.

� BIP 2066 : Broker <insert_0> (UUID <insert_1>) was unable to retrieve an internal configuration response message for execution group '<insert_2>' within the <insert_3> second Configuration Timeout.

� BIP 2087 : Broker <insert_0> was unable to process the internal configuration message.

� BIP 1536 : The Configuration Manager was unable to register for internal subscriptions with broker <insert_0>.

IBM Software Group


Notes (No Response):

� BIP 2070 : A problem was detected with WebSphere MQ while issuing <insert_0> for WebSphere MQ queue '<insert_1>', WebSphere MQ queue manager '<insert_2>'. MQCC=<insert_3>, MQRC=<insert_4>.

� BIP2322 : Database error: SQL State '<insert_0>'; Native Error Code '<insert_1>'; Error Text '<insert_2>'.

IBM Software Group


Notes (Debugger not working):

� RAC log: What we usually see is : � error=53, which on AIX means : Unable to create a socket So RAC

is unable to open a socket to the XP client usually it is because of firewall, filter or anything on the network

IBM Software Group


Notes (Support Website):

You should start your search with minimal search terms and special keywords. Then, to reduce the number of item returned, add more specialized keywords. Search terms are not case sensitive Special words like OR and NOT must be capitalized. For example

mqsi* wbi* R500 JVM "deploy failure" NOT toolkit

IBM Software Group


Notes (Additional APARS)

APAR IC48712A breakpoint is set between two nodes in the Toolkit debugger. When the variables associated with the message

are displayed, the user wishes to change the value of a variable. The variable is right clicked and the option to

change the variable is selected . OK is now selected (after the variable is changed or it can be left unchanged) and the Toolkit disconnects.

APAR JR23282Ariba Buyer does not have verbs for child objects. For this reason, the adapter was not designed to populate the

verb value for child objects. The xsd generated by the adapter framework though requires verbs for all objects.APAR IC48399A user has their main map in one broker schema and submaps in another broker schema (which is not the default

schema). When creating a mapping, the user opts to call an existing submap. The tool generates the submapcall for the user, but the call doesn't include the broker schema of the submap. This results in an unresolved

function call. The code for generating the submap call has been modified so that it no longer assumes thatsubmaps always reside in the same broker schema as the caller.

IBM Software Group


Notes (Additional APARS)

APAR IC48389

The source and target are defined in two separate message sets. Both source and target have their own global types. The type of the source and type of the target have the same type name but in different namespace. In both message sets, there is no prefix defined for the namespace of the types. In these cases the tool thought the two types were identical, it therefore created a deep copy from source to target. The two types are actually different, and a submap call should be created. The code used to determine if two types are the same has been modified so that namespace prefix, or the lack of it, are taken into consideration.

APAR IC48373

The debug status of a message flow in the toolkit is not preserved when the flow is deployed again. This has now been corrected and the debug status is now correctly passed on to the database.

wmb topten problems

Documents

broker queue manager

broker box

mq seriescheck mq logs

ibm software groupmichael

message logbroker restart

time messages

configuration delay

bip2066e configuration