datastage stages
TRANSCRIPT
![Page 1: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/1.jpg)
DIFFERENT STAGESDIFFERENT STAGES
![Page 2: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/2.jpg)
Aggregator Stage :
Definition : Aggregator classifies data rows from a single input link into groups and calculates totals or other aggregate functions for each group. The summed totals for each group are output from the stage thro' output link.
Group is a set of record with the same value for one or more columns.
Example : Transaction records might be grouped by both day of the week and by month. These groupings might show the busiest day of the week varies by season.
![Page 3: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/3.jpg)
Input & View data :
The INPUT page shows you the metadata of the incoming data.
The input data look like this…
![Page 4: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/4.jpg)
Properties :
Here, we group by "Gender".
The column to aggregate.
User defined column to collect the aggregated values.
When "Aggregation Type = Calculation" …
![Page 5: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/5.jpg)
Output & View data :
The OUTPUT page shows only those columns used to group and aggregate.
As we have grouped by "Gender", the incomes of Males and Females are summed up and shown here.
![Page 6: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/6.jpg)
Execution Mode :
Note :
The Aggregator stage can have only one output link.
![Page 7: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/7.jpg)
Properties :
Here, we group by "Gender".
The column to be counted.
When "Aggregation Type = Count Rows" …
![Page 8: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/8.jpg)
Output & View data :
The OUTPUT page shows only the grouping column and the column to be counted.
As we have grouped by "Gender", the number of records in Males and Females are totaled and shown here.
![Page 9: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/9.jpg)
Execution Mode :
Note :
The Aggregator stage can have only one output link.
![Page 10: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/10.jpg)
Properties :
Here, we group by "Gender".
The column to preserver the summary of Recalculation.
When "Aggregation Type = Re-calculation" …
Note :
When the "Aggregation Type = Re-calculation" then place an extra aggregator to aggregate a column, first. The second aggregator will re-calculate the previously
calculated column.
![Page 11: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/11.jpg)
Output & View data :
The OUTPUT page shows only the grouping column and the column for recalculation.
The column "MaxVal” is recalculated as "SumOfMaxVal"
![Page 12: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/12.jpg)
Execution Mode :
Note :
The Aggregator stage can have only one output link.
![Page 13: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/13.jpg)
Change Apply Stage :
Definition : Takes the change data set, that contains the changes in the before and after data sets, from the Change Capture stage and applies the encoded change operations to a before data set to compute an after data set.
The Change Apply stage read a record from the change data set and from the before data set, compares their key column values, and acts accordingly.
![Page 14: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/14.jpg)
Change Capture Property :
Change Apply Property :
![Page 15: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/15.jpg)
Input (Before Changes) :
Input (After Changes) :
Output :
![Page 16: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/16.jpg)
Job :
![Page 17: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/17.jpg)
Filter Stage :
Definition : The Filter stage transfers, unmodified, the records of the input data set which satisfy the specified requirements and filters out all other records.
Filter stage can have a single input link and a any number of output links and, optionally, a single reject link. You can specify different requirements to route rows down different output links. The filtered out records can be routed to a reject link, if required.
![Page 18: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/18.jpg)
Simple Job :
![Page 19: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/19.jpg)
Criteria to Filter :
Note : Only if the "Output Rejects=True" the rejected rows are collected separately, otherwise those rows that fails to satisfy the criteria will be ignored.
![Page 20: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/20.jpg)
Input :
Output : Reject Rows :
Criteria : Salary>30000
![Page 21: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/21.jpg)
Complex Job :
![Page 22: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/22.jpg)
Criteria to Filter :
Note : "Output Row Only Once=False" means, every single input row is forced to satisfy each and every criteria given. In other words, Row that satisfies a criteria is forced to undergo another criteria. In such case, every row gets more than a single chance to output.
![Page 23: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/23.jpg)
Input Data
Criteria 1 : Salary>30000 Criteria 2 : Number>1
Criteria 3 : Number=3 Reject Rows :
![Page 24: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/24.jpg)
Criteria to Filter :
Note : "Output Row Only Once=True" means, every single input row is not forced to undergo each and every criteria given. In other words, Rows that satisfy at least one criteria is not forced to satisfy another criteria. In such case, every row gets a single chance to output.
![Page 25: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/25.jpg)
Input Data
Criteria 1 : Salary>30000 Criteria 2 : Number>1
Criteria 3 : Number=3 Reject Rows :
Note : Though there is a row that satisfies this criteria, it is not outputted as it is already been outputted for satisfying the Criteria – 1.
![Page 26: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/26.jpg)
Funnel Stage :
Definition : Funnel Stage copies multiple input data sets to a single output data set. This operation is useful for combining separate data sets into a single large data set. The stage can have any number of input links and a single output link.
![Page 27: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/27.jpg)
![Page 28: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/28.jpg)
Funnel Stage :
Definition : Funnel Stage copies multiple input data sets to a single output data set. This operation is useful for combining separate data sets into a single large data set. The stage can have any number of input links and a single output link.
![Page 29: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/29.jpg)
![Page 30: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/30.jpg)
Type – 1 : Continuous Funnel
Note : This type of Funnel combines the records of the input data in no guaranteed order. It takes one record from each input link in turn. If data is not available on an input link, the stage skips to the next link rather than waiting.
![Page 31: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/31.jpg)
Output view data :
Input - 1 view data : Input - 2 view data :
Continuous Funnel type …
![Page 32: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/32.jpg)
Type – 2 : Sequence
Note : This type of Funnel copies all records from the first input data set to the output data set, then all the records from the second input data set, and so on.
![Page 33: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/33.jpg)
Output view data :
Input - 1 view data : Input - 2 view data :
Sequence type …
![Page 34: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/34.jpg)
Type – 3 : Sort Funnel
Note : This type of Funnel combines the input records in the order defined by the value(s) of one or more key columns and the order of the output records is determined by these sorting keys.
![Page 35: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/35.jpg)
Output view data :
Input - 1 view data : Input - 2 view data :
Sort Funnel type …
![Page 36: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/36.jpg)
Job :
![Page 37: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/37.jpg)
Join Stage :
Definition : Join Stage performs join operations on two or more data sets input to the stage and then outputs the resulting data set.
The input data sets are notionally identified as the "right" set and the "left" set, and "intermediate" sets. It has any number of input links and a single output link.
![Page 38: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/38.jpg)
Join Type = Full Outer
Job :
![Page 39: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/39.jpg)
Left Input : Right Input :
Output :
![Page 40: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/40.jpg)
Join Type = Inner
Job :
![Page 41: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/41.jpg)
Left Input : Right Input :
Output :
![Page 42: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/42.jpg)
Join Type = Left Outer
Job :
![Page 43: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/43.jpg)
Left Input : Right Input :
Output :
![Page 44: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/44.jpg)
Join Type = Right Outer
Job :
![Page 45: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/45.jpg)
Left Input : Right Input :
Output :
![Page 46: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/46.jpg)
Lookup Stage :
Definition : Lookup Stage used to perform lookup operations on a data set read into memory from any other Parallel job stage that can output data.
It can also perform lookups directly in a DB2 or Oracle database or in a lookup table contained in a Lookup File Set stage.
![Page 47: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/47.jpg)
Mappings :
![Page 48: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/48.jpg)
Input :
References :
Output :
Reject :
![Page 49: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/49.jpg)
Job :
![Page 50: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/50.jpg)
Merge Stage :
Definition : Join Stage combines a sorted master data set with one or more update data sets. The columns from the records in the master and update data sets are merged so that the output record contains all the columns from the master record plus any additional columns from each update record.
A master record and an update record are merged only if both of them have the same values for the merge key column(s) that you specify. Merge key columns are one or more columns that exist in both the master and update records.
The data sets input to the Merge stage must be key partitioned and sorted. This ensures that rows with the same key column values are located in the same partition and will be processed by the same node.
![Page 51: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/51.jpg)
Unmatched Masters Mode = Drop
Job :
![Page 52: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/52.jpg)
Master :
Updates :
Output :
Reject :
![Page 53: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/53.jpg)
Job :
Unmatched Masters Mode = Keep
![Page 54: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/54.jpg)
Master :
Updates :
Output :
Reject :
![Page 55: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/55.jpg)
Note : If the options "Warn on Reject Updates = True" and "Warn on Unmatched Masters = True" then the log file shows the warnings on Reject Updates and Unmatched Data from Masters.
![Page 56: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/56.jpg)
Note : If the options "Warn on Reject Updates = False" and "Warn on Unmatched Masters = False" then the log file do not shows the warnings on Reject Updates and Unmatched Data from Masters.
![Page 57: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/57.jpg)
Modify Stage :
Definition : The Modify stage alters the record schema of its input data set. The modified data set is then output. It is a processing stage.
It can have a single input and single output.
![Page 58: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/58.jpg)
Job (before handling):
Null Handling:Null Handling:
![Page 59: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/59.jpg)
Null Handling…
Step – 1:
"NULL" value has to be handled…
![Page 60: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/60.jpg)
Step – 2:
Null Handling
Syntax : Column_Name=Handle_Null('Column_Name',Value)
Input Link Columns Output Link Columns
Null Handling…
![Page 61: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/61.jpg)
Step – 3:
"NULL" value has been handled…
Null Handling…
![Page 62: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/62.jpg)
Job (after execution):
Null Handling…
![Page 63: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/63.jpg)
Job (before execution):
Drop Column(s):Drop Column(s):
![Page 64: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/64.jpg)
Step – 1:
The column "MGR" has to be dropped…
Drop Columns …
![Page 65: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/65.jpg)
Step – 2:
Dropping Column
Syntax : DROP Column_Name
Input Link Columns Output Link Columns
Drop Columns …
![Page 66: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/66.jpg)
Step – 3:
The column "MGR" has
been dropped…
Drop Columns …
![Page 67: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/67.jpg)
Job (after execution):
Drop Columns …
![Page 68: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/68.jpg)
Job (before execution):
Keep Column(s):Keep Column(s):
![Page 69: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/69.jpg)
Step – 1:
The column "EmpNo" has to be kept…
Keep Columns …
![Page 70: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/70.jpg)
Step – 2:
Keeping Column
Syntax : KEEP Column_Name
Input Link Columns Output Link Columns
Keep Columns …
![Page 71: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/71.jpg)
Step – 3:
The column "EmpNo" is
kept…
Keep Columns …
![Page 72: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/72.jpg)
Job (after execution):
Keep Columns …
![Page 73: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/73.jpg)
Job (before execution):
Type Conversion:Type Conversion:
![Page 74: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/74.jpg)
Step – 1:
The column "HireDate" has to
converted to Date…
Type Conversion …
![Page 75: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/75.jpg)
Step – 2:
Type Conversio
n
Syntax : Column_Name=type_conversion('Column_Name')
Input Link Columns Output Link Columns
Type Conversion …
![Page 76: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/76.jpg)
Step – 3:
The column "HireDate" has
been converted to Date…
Type Conversion …
![Page 77: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/77.jpg)
Job (after execution):
Type Conversion …
![Page 78: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/78.jpg)
Job (before execution):
Multiple Specifications:Multiple Specifications:
![Page 79: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/79.jpg)
Step – 1:
The column "HireDate"
has to converted to
Date…
The column "MGR" has to
be Null handled…
Multiple Specification …
![Page 80: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/80.jpg)
Step – 2:
Null Handling
Type Conversio
n
Input Link Columns Output Link Columns
Multiple Specification …
![Page 81: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/81.jpg)
Step – 3:
The column "HireDate" has been
converted to Date…
The column "MGR" has been Null handled…
Multiple Specification …
![Page 82: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/82.jpg)
Job (after execution):
Multiple Specification …
![Page 83: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/83.jpg)
Pivot Stage converts columns in to rows.
Eg., Mark-1 and Mark-2 are two columns.
Task : Convert all the columns in to one column.
Implication : Can be used to co SCD Type-3 to Type-2.
Using Methodology : In the deviation field of the output column change the input columns in to one column.
Eg., Column Name – "Marks".
Derivation : Mark-1 and Mark-2.
Note : Column "Marks" is derived from the input columns Mark-1 and Mark-2.
Pivot Stage :
![Page 84: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/84.jpg)
Job (before execution):
![Page 85: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/85.jpg)
Source File
Input metadata
Output Metadata:
Note the change in the derivation …
![Page 86: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/86.jpg)
Job (after execution) :
Output file:
![Page 87: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/87.jpg)
Remove Duplicates Stage:
Definition : The Remove Duplicates stage takes a single sorted data set as input, removes all duplicate records, and writes the results to an output data set.
Removing duplicate records is a common way of cleansing a data set before you perform further processing. Two records are considered duplicates if they are adjacent in the input data set and have identical values for the key column(s).
![Page 88: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/88.jpg)
Last duplicate row dropped…
Output view data :Input view data :
Selecting Key & Retrain Row : Sorting Column :
![Page 89: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/89.jpg)
First duplicate row dropped…
Output view data :
Selecting Key & Retrain Row :
Input view data :
![Page 90: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/90.jpg)
Job (after execution) :
![Page 91: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/91.jpg)
Surrogate Key Generator Stage :
Definition : The Surrogate Key stage generates key columns for an existing data set.
User can specify certain characteristics of the key sequence. The stage generates sequentially incrementing unique integers from a given starting point. The existing columns of the data set are passed straight through the stage.
If the stage is operating in parallel, each node will increment the key by the number of partitions being written to.
![Page 92: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/92.jpg)
Job (before execution):
![Page 93: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/93.jpg)
Input :
Output :
Property :
![Page 94: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/94.jpg)
Job (after execution):
![Page 95: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/95.jpg)
Switch Stage :
Definition : The switch stage takes a single data set as input and assigns each input row to an output data set based on the value of a selector field.
It can have a single input link, up to 128 output links and a single rejects link. This stage performs an operation similar to a C switch statement. Rows that satisfy none of the cases are output on the rejects link.
![Page 96: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/96.jpg)
Property :
![Page 97: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/97.jpg)
Input :
Output - 1 :
Reject :
Output - 2 :
Output - 3 :
![Page 98: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/98.jpg)
Job (after execution):
![Page 99: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/99.jpg)
Property :
![Page 100: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/100.jpg)
Input :
Output - 1 :
Output - 2 :
Output - 3 :
![Page 101: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/101.jpg)
Job (after execution):
![Page 102: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/102.jpg)
Property :
![Page 103: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/103.jpg)
Input :
Output - 1 :
Reject :
Output - 2 :
Output - 3 :
![Page 104: DataStage Stages](https://reader033.vdocuments.site/reader033/viewer/2022061505/55cf94cd550346f57ba47b27/html5/thumbnails/104.jpg)
Job (after execution):