workflow manager and general tuning tips. topics to discuss… working with workflows working with...
TRANSCRIPT
Workflow Manager and General Tuning Tips
Topics to discuss…
Working with Workflows Working with Tasks General Tuning Tips
Working with Workflows
A workflow is a set of instructions that tells the
PowerCenter Server how to execute tasks such
as sessions, email notifications, and shell
Commands.
Working with Workflows
Developing New Workflows.
Everybody has created more than 1 workflow !!
Working with Workflows - Links
Working with Links Use links to connect each workflow task. You can specify conditions with links
to create branches in the workflow. The Workflow Manager does not allow you to use links to create loops in the workflow. Each link in the workflow can execute only once.
a valid workflow:
Working with Workflows - Links
The Workflow Manager does not allow you to create a workflow that contains a loop
Invalid workflow:-
Working with Workflows - Links
Specifying Link Conditions
Once you create links between tasks, you can specify conditions for each link to determine the order of execution in the workflow. If you do not specify conditions for each link, the PowerCenter Server executes the next task in the workflow by default.
You can use pre-defined or user-defined workflow variables(?) in the link condition. If the link condition evaluates to True, the PowerCenter Server executes the next task in the workflow. If the link condition evaluates to False, the PowerCenter Server does not execute the next task in the workflow.
You can view results of link evaluation during workflow runs in the workflow log file.
Working with Workflows - Variables
Pre-defined workflow variables. The Workflow Manager provides pre-defined workflow variables for tasks within a workflow.
User-defined workflow variables. You create user-defined workflow variables when you create a workflow.
Working with Workflows - Variables
Pre-defined workflow variables1. Task-specific variables 2. System variables
-Condition -SrcSuccessRows -SYSDATE
-EndTime -StartTime -WORKFLOWSTARTTIME
-ErrorCode -Status
-ErrorMsg -TgtFailedRows
-FirstErrorCode -TgtSuccessRows
-FirstErrorMsg -TotalTransErrors
-PrevTaskStatus
-SrcFailedRows
Working with Workflows - Variables
User-defined workflow variables
Working with Workflows - Variables
Start and current value:-– Value in parameter file – Value saved in the repository (if the variable is persistent) – User-specified default value – Datatype default value
Datatype Workflow Manager Default Value
Date/time 1/1/1753 A.D.Double 0Integer 0Nstring Empty string
Working with Workflows - Scheduler
You can schedule a workflow to run continuously, repeat at a given time or
interval, or you can manually start a workflow. The PowerCenter Server runs a
scheduled workflow as configured.
By default, the workflow runs on demand. You can change the schedule settings
by editing the scheduler. If you change schedule settings, the PowerCenter Server
reschedules the workflow according to the new settings.
When a workflow fails, the PowerCenter Server removes the workflow from the
schedule, and you must manually reschedule it.
Working with Workflows - Scheduler
Working with Workflows - Suspending
When a task in the workflow fails, you might want to suspend the workflow, fix the
error, and resume or recover the workflow. The PowerCenter Server suspends the
workflow if you enable the Suspend On Error option in the workflow properties.
You can optionally set a suspension email so the PowerCenter Server sends an
email when it suspends a workflow.
When you enable the Suspend On Error option, the PowerCenter Serversuspends the workflow when one of the following fails:
– Session – Command – Worklet – Email
Questions
?
Working with Tasks
Assignment
Command
Control
Decision
Event-Raise
Event-Wait
Session
Timer
Working with Tasks - Assignment
The Assignment task allows you to assign a value to a user-defined workflow variable. To use an Assignment task in the workflow, first create and add the Assignment task to the workflow. Then configure the Assignment task to assign values or expressions to user-defined variables. After you assign a value to a variable using the Assignment task, the PowerCenter Server uses the assigned value for the variable during the remainder of the workflow.
Working with Tasks - Commands
If you select the Run If Previous Completed option, when one of the commands in the Command task fails, the PowerCenter Server stops running the rest of the commands and fails the task. If you do not select the Run If Previous Completed option, the PowerCenter Server runs all the commands in the Command task and treats the task as completed, even if a command fails.
Working with Tasks - Example
Example:-
Working with Tasks - Control
You can use the Control task to stop, abort, or fail the top-level workflow or the parent workflow based on an input link condition. A parent workflow or worklet is the workflow or worklet that contains the Control task.
Working with Tasks - Control
Example:-This can be used in a worklet. And the condition given in the control task will be Stop top-level workflow. So whenever the decision routs the flow towards the control task, an email will be generated and then the workflow containing the below worklet will stop and not proceed further.
Working with Tasks - Decision
Nothing new under the sun
Working with Tasks - Events
Event-Raise task. Event-Raise task represents a user-defined event. When the PowerCenter Server runs the Event-Raise task, the Event-Raise task triggers the event. Use the Event-Raise task with the Event-Wait task to define events.
Event-Wait task. The Event-Wait task waits for an event to occur. Once the event triggers, the PowerCenter Server continues executing the rest of the workflow.
Pre-defined event User-defined event.
Working with Tasks - Events
Pre-defined event. A pre-defined event is a file-watch event. For pre-defined events, use an Event-Wait task to instruct the PowerCenter Server to wait for the specified indicator file to appear before continuing with the rest of the workflow. When the PowerCenter Server locates the indicator file, it starts the next task in the workflow.
User-defined event. A user-defined event is a sequence of tasks in the workflow. Use an Event-Raise task to specify the location of the user-defined event in the workflow. A user-defined event is sequence of tasks in the branch from the Start task leading to the Event-Raise task.
Working with Tasks - Events
Example – Used Defined Event !
Questions
?
Performance Tuning
Identifying the Performance Bottleneck
1>Identifying Target Bottlenecks
2>Identifying Source Bottlenecks
- Using a Filter Transformation- Using a Read Test Session - Using a Database Query
3>Identifying Mapping Bottlenecks
- High Rowsinlookupcache Counters- High Errorrows Counters
4>Identifying a Session BottleneckAny value other than zero in the readfromdisk and writetodisk counters for Aggregator,
Joiner, or Rank transformations indicate a session bottleneck.
5>Identifying a System Bottleneck
Performance Tuning
Optimizing the Target Database – Drop indexes and key constraints. – Increase checkpoint intervals. – Use bulk loading. – Use external loading. – Increase database network packet size. – Optimize Oracle target databases.
Performance Tuning
Optimizing the Source Database – Optimize the query. – Create tempdb as in-memory database. – Use conditional filters. – Increase database network packet size. – Connect to Oracle databases using IPC protocol.
Performance Tuning
Optimizing the Mapping- Configure single-pass reading.
- Optimize datatype conversions.
- Eliminate transformation errors.
- Optimize transformations.
- Reduce the number of transformations.
- Optimize expressions.
- Figure out common logic and place in a variable flag instead of calculating each time.
Performance Tuning
Optimizing Lookup TransformationsCaching Lookups
- Shared cache.
- Persistent cache
Reducing the Number of Cached Rows
Optimizing the Lookup Condition
Indexing the Lookup Table
Performance Tuning
Optimizing the Session
– Increase the number of partitions. – Reduce errors tracing. – Remove staging areas. – Tune session parameters.
Performance Tuning
Optimizing the SystemImprove network speed.
Use multiple PowerCenter Servers.
Use a server grid.
Improve CPU performance.
Configure the PowerCenter Server for ASCII data movement mode.
Check hard disks on related machines.
Reduce paging.
Use processor binding
Questions
?
Thank you !!!