© 2014 IBM Corporation

Effective administration in IBM Integration Bus (1139)Dave GormanIBM Corporationgormand@uk.ibm.com

Please Note

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

What will this session cover?

WMB and IIB have support for two main user roles• Integration Developer• Administrator

This session will aim to cover accepted best practice for the most typical administrator tasks.• Including some essential “top tips”

Topics to discuss• Tools of the trade• Administrative tasks• Under the hood

I’ll focus on WMB V8• but will point out some IIB V9 improvements too!

Interaction With Tools

IntegrationToolkit Command lineCommand line

Integration Node

Third Party Third Party ToolsTools

Integration API (CMP)

REST API

IntegrationExplorer

WebAdmin

REST API WASAdmin

Integration Toolkit View for Developers

Advanced management option designed for administrat ors

Plug-in to MQ Explorer

Extra features

− Create/Manage Configurable Services

− Performance Views

− Group integration nodes using sets

− Offload WS-Security onto Datapower

− Administration Log

− Administration Queue

− Manage statistics & tracing

Integration Explorer

Command line tools

A wide selection of tools for scripting actionsRequires a configured environment• mqsiprofile or mqsicommandconsole (Windows)

mqsiprofile (Linux/UNIX) JCL or ISPF (z/OS)

Most commands work against local or remote integrat ion nodes

BIP1121I: Creates an execution group.

Syntax:mqsicreateexecutiongroup brokerSpec -e egName [-w timeoutSecs] [-v traceFileName]

Command options:'brokerSpec' is one of:

(a) 'brokerName' : Name of a locally defined broker(b) '-n brokerFileName' : File containing remote br oker connection parameters (*.broker)(c) '-i ipAddress -p port -q qMgr' : hostname, port and queue manager of a remote broker

'-e egName' name of the new execution group'-w timeoutSecs' maximum number of seconds to wait for the execution group to be created'-v traceFileName' send verbose internal trace to t he specified file.

Integration API (CMP)

Java interface that enables the administration tool sUse for custom administration requirementsFully documented and samples available

• MB v8 and later allows you to create and edit messa ge flows too– Build your entire system programmatically!

8

Compelling choice for WebSphere Application Server• WAS provides efficient application development and delivery• Simplifies learning curve for WAS users• Addresses administrator requirements• Supported on WAS V7 and V8

WAS Admin Console Plug-in• Varied set of administration tasks available

–Connect to multiple local or remote integration nodes–View available integration servers and their current status–View services, applications, libraries, message flows

• Uses standard features for ease of configuration–Role-based access to prevent unauthorized administration–Uses REST APIs for local and remote management–Use WAS Admin Console for WAS centric administrator–Complements Web UI for the administrator

• Design allows for future modification of resources, start, stop etc.

Fully compatible with WAS ND• In plugin operates at cell level.• The configuration data is stored by the Deployment manager.• All Application Servers have access to cell level configuration data.

Natural integration with WAS Admin Console

9

Web Visualisation and Analytics� A comprehensive tool for web management

– Manage all integration resources from zero-footprint client

– Analyze integration performance in real-time– Supported on a variety of browsers: IE10, Firefox, Safari…– Complements MQ Explorer and WAS Admin consoles

� Managing Integration Resources– View top-level integration node properties– Add/remove/change integration servers– Start/Stop integration data flows

– Role based access to control usage– Advanced options include data replay, policy & monitoring– Exploits underlying public REST/JSON API

� Integration Performance Analysis– Operational experience; no developer intervention required

• New and existing flows can exploit without change

– Many metrics of integration flow available in real-time• CPU & I/O time shown by default in integration analyzer

• Other metrics include thread, data sizes, errors…

– Flexible display includes data tables and flow profile• Drill down to understand detailed behaviour

– Exploits underlying MQTT web sockets technology• Asynchronous notification at low CPU cost

Development Lifecycle and Environments

• Integration Developer– Develops message flows,

message models etc.

– Unit Tests on local machine

– Creates archive (BAR) files containing required artefacts

• Administrator– Customizes BAR for target

environment (message flow properties including queues, database names etc.)

– Deploys BAR to target environment

– Management and operational control

Development Test QA Production

Development Test & QA Production

or

Common Administrative Tasks

Planning and configuration• Bringing a new integration node online• Securing an integration node

• Making an integration node highly available

• Planning for disaster recovery

Managing integration nodes• Managing what’s deployed

• Understanding behaviour• Optimizing and tuning

• Creating/Setting Policies

• Migration• Maintenance

Bringing a new integration node onlineTypical steps required• Preconfiguration (e.g. OS, userids)• Installation (MQ, IIB, additional software)

• Creating the integration node

• Creating integration servers• Creating Configurable Services

• Deploying BAR files

• Additional configuration (e.g. security, userids)

Ensure that the environment is reproducible• Documentation

• Scripting• Virtualization

Cloud IntegrationConsider virtualization for new environments• Makes it easy to provision systems (and restore known state!)• WMB and IIB fully supports the use of them• Understand maintenance and performance aspects

WMB / IIB Hypervisor EditionsChef cookbooks for simplified IIB provisioningIBM Pure Application Systems• pre-optimized, ready-to-go systems• Quickly provision diverse product combinations WAS, MQ, MB etc…

IBM Workload Deployer appliance• Virtual image software & hardware packages

– RHEL 6.2 for VMware ESX x86-64 & IWD images– AIX 6.1 TL7 mksysb & IWD images

deployconfigure

HVEConfigpattern

Securing WMB and IIB

Simplified administrative security3 levels of authorisation for administrative action s• Reading (e.g. View integration node properties - mqsilist)• Writing (e.g. Change integration node properties -

mqsicreateexecutiongroup)• Executing (i.e. starting and stopping) • See topic bp43540 for more detailsOn two object types:• “Integration node” and “Integration servers”Administrative Security is not enabled by default• Enable through mqsichangebroker / mqsicreatebrokerAccess controlled using MQ queues on the integratio n node’s queue manager• Give authority to users or groups

+inq = Read

+put = Write

+set = Execute

Security Queues SYSTEM.BROKER.AUTH

SYSTEM.BROKER.AUTH.<egname>

Making the integration node highly available

How do I ensure that the integration node is contin ually processing messages ?• Active/Active vs. Active/Passive

• Agree SLAs with the business (% uptime)

The integration node includes restart recovery of i ntegration servers• But this is usually not sufficient on its own• Does not cover machine restart

Two main options• Third-party solutions (e.g. VCS/HACMP/MSCS)

• Multi-instance queue managers and integration nodes

Distributed Cache• Built in cache support in 8.0.0.1 and later. • Enables HA scenarios

Planning for disaster recovery

What would you do if your primary WMB location goes down?

Distribute WMB to multiple sites if possible• This introduces data replication and latency concerns

Keep DR concerns separate from HA!• HA: Systems at a single site with a single configuration

DR: Systems at multiple sites with replicated configurations

• An HA failover can be a planned activity

• DR is unplanned.

Backing Up

Regularly use mqsibackupbroker to backup the integration node’s configuration• Ideally, after all configuration changes

• Best run when stopped• But can be used whilst active as long as not undergoing configuration

changes

Consider any other required resources, for example:• Database tables

• Source artefacts (message flows, BAR files)

Ensure regular restore testing• Prove your process works

Pattern to reconstitute message flows from a runnin g integration node• Available on MQSeries.net

Managing what’s deployed

• It can be difficult to understand what individual d eployed resources are used for

• Applications and libraries can allow you to underst and why each file is there– Application: Encapsulates a single use case or scenario

– Library: Promotes re-use of a shared set of files

• Concepts are shared between developers and administ rators– The developer chooses to create an application or library; the collection is then carried all the way

through to the runtime

Managing shared resources

V8 and later also allow you to deploy source files directly to the runtime• Subflows (.subflow), ESQL (.esql) … in addition to CMF etc.

Allows you to manage subflows and ESQL independently of the message flows that use them

After deployment of a new version, all affected mes sage flows pick up any changes with the next message

.esql

.msl

New!

.dfdl

New!

.subflow

New!

BAR

Integration Node

Enables record and replay in-flight data

Universal support for file, messaging, web service, ERP & other data

Multiple data formats (binary, text, XML) WebUI tooling for access to end users, auditors and administrators

Managing Record & Replay

The tools include a lot of information that is usef ul to the administrator, for example:• Administration queue and log, Message flow and resource

statistics• Warnings when features that affect performance are enabled

Understanding integration node behaviour

• Use this information to understand recent configura tion changes

− How the integration server is performing, connected endpoints, etc.

The Activity Log shows you all recent activity on a message flow or resource manager• For example, “Show me all recent JMS activity”

Visible in Integration Explorer and/or written to a fileCustomisable rotation rules (based on age or size) and content

Understanding integration node behaviour

Optimizing and tuning

The tools allow you to modify the configuration ope rationally These tweaks more efficient to make than modifying message flowsEncourage developers to create message flows that e nable operational tweaks to be made, e.g.• Policy

• Configurable Services

• User-defined properties

• User-defined configurable services

25

Controlling Integrations with Policy� Integration Workload Management

– Provide intelligent mechanisms to control processing speed– Most common scenario is to reduce back-end server load

– Design allows more policy-based processing over time– Can be applied to new or existing integration data flows

� Policy defines threshold limits and relevant action s– Set thresholds for integration data flow throughput– Specify actions at threshold, for example:

• NOTIFY: Higher (or lower) than threshold generates publication

• DELAY: Excessive workload will have latency added to shape throughput

� Web Console used to manage WLM policy– Sophisticated behaviour controllable by WLM policy

– Workload can be managed across classes of message flows (e.g. batch vs. online)– Policies stored in local registry, and dynamically configurable– Developer can also specify limits as integration data flow properties

26

� Target unresponsive flows through policy to improve overall system reliability– Additional WLM option aimed at unresponsive integration flows– An integration flow can become unresponsive for multiple reasons

• e.g. Waiting for external system, infinite loop, deadlock, malformed XML

� Flexible configuration, actions and reporting optio ns– Specify threshold at which flows are considered unresponsive, e.g. 30 seconds for processing

• Configured via WLM policy, or directly on the flow in the BAR file

– Define action to trigger when flow considered unresponsive• Administrative notification through a new “timeout exceeded” event message

– If flow eventually continues through to completion, a second event is published

• Restart the integration server (execution group) on which the unresponsive flow is running

– New command option to forcibly stop integrations manually: mqsistopmsgflow –f

Managing Unresponsive Integration Flows

Waiting for response

27

432

Migration from WMB V6.1, V7 and V8� Migration from WMB V6.1, V7 and V8

– All development assets (e.g. message flows, ESQL, DFDL, Java, Maps and XSLT) import directly

• Right-click convert action for pre-V8 maps; some manual tasks may be required

– Migrate brokers using a single command, or create new integration nodes for phased migration

• No redeployment necessary when using built-in migrate command• All existing BAR files can be deployed to IB V9 integration nodes without change

� Migration commands for in-place migration– Includes migration of configuration data including databases, queues and registry– Forwards and backwards migration of existing components, in situ

• mqsimigratecomponents command (includes –t option for rollback to V7 and V8)

� Flexible co-existence options remove the need for a dditional hardware when migrating– IB V9 co-exists on the same OS with all previous MB versions

– MQ V7.5.0.1 required for all IB V9 integration nodes• MQ V7.5.01 supported with V7 and V8 brokers for the purposes of V9 migration• For V6.1 migration, upgrade MQ and MB simultaneously

1 Install IB V9 Stop broker Run migrate command Start broker

4321 Install IB V9 Create new broker Deploy existing assets Stop old broker

or

Maintenance

Schedule regular maintenance windows• IBM recommends that you are on the latest maintenance level• Plan exactly what will be applied and when• A highly available environment ensures that there is no downtime

IIB has an option to enable/disable new capability in fixpacks• New functionality disabled by default

� New nodes and parsers are disabled by default

Summary

We have discussed a number of different ways that h elp ensure that administration is trouble-free

My Top Tips• Always ensure your environment is reproducible• Treat DR and HA separate• Ensure regular backups• Encourage developers to create message flows that enables

operational tweaks to be made• Schedule regular maintenance windows

Questions?

We Value Your Feedback

Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference.Use the Conference Mobile App or the online Agenda Builder to quickly submit your survey

• Navigate to “Surveys” to see a view of surveys for sessions you’ve attended

31

IBM Integration Bus This Week – Selected SessionsMonday

• 2.30-3.30 Palazzo H – Integration Featured Session• 2.30-5.00 Murano 3305 – Healthcare Integration Lab• 4.00-5.00 Palazzo H – What's New in IIB• 5.15-6.15 Palazzo H – Introduction to IIB

Tuesday• 10.30-11.30 Palazzo H – Cloud Integration Options• 10.30-11.30 San Polo 3502 – Meet The Experts• 1.00-2.00 Palazzo H – WESB Conversion• 1.00-2.00 Marcello 4403 – IIB Retail Integration Pack• 1.00-3.15 Murano 3303 – IIB V9 Lab• 2.15-3.15 Palazzo H – Designing for Performance• 3.45-4.45 Palazzo H – Mobile Integration• 5.00-6.00 Palazzo H – DFDL Introduction• 5.00-6.00 Marcello 4402 – Manufacturing in IIB

32

IBM Integration Bus This Week – Selected Sessions

Wednesday• 10.30-11.30 Palazzo H – .NET Integration• 1.00-2.00 Palazzo H – Effective Application Development• 1.00-2.00 Marcello 4403 – IIB Healthcare Integration Pack• 1.00-2.00 San Polo 3503 – Meet The Experts (Repeat)• 2.15-3.15 Palazzo H – Effective Administration• 3.45-4.45 Palazzo H – Applications, Libraries, APIs• 3.45-6.00 Murano 3303 – IIB Open Beta Lab• 5.00-6.00 Palazzo H – Transformation Options in IIB

Thursday• 9.00-10.00 Palazzo H – Predictive Analytics• 10.30-11.30 Palazzo H – BPM Integration• 1.00-2.00 Palazzo H – Modelling Industry Formats• 2.15-3.15 Palazzo H – What's New in IIB (Repeat)

33

Thank You

Legal Disclaimer

• © IBM Corporation 2014. All Rights Reserved.• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained

in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in yourpresentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States andother countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.