defining components

8/12/2019 Defining Components

1/53

Defining components

Composition view of data

Double-click group object in Type tree view to define and view group components in the

order they appear in data in group view

Defining components

Identify the data types that make up a group

Describe the order that data appears in a group


2/53

Component lists

Component name in group view refers to the type in the type tree using the full name

Always drag and drop types into component lists


3/53

Component lists

Component ranges

Component range indicates the number of consecutive occurrences of component allowed

Specified as minimum_occurrences:maximum_occurrences

The default is one and only one, designated by 1:1

The range of 1:1is assumed if no value is displayed

If the component is optional,

minimum occurrences is

zero, designated by 0:1

If component can appear an

unknown number of times;

the maximum_occurrences is S


4/53

Displaying all range values

Setting the range

Type tree analyzer

Logical analysis verifies integrity of defined relationships

Detects undefined components

Identifies distinguishability problems

Structure analysis verifies integrity of physical type tree file


5/53

Does not compare definitions to actual data

Type tree must analyze error-free before it can be used in map

All errors must be resolved

Warnings indicate an inconsistency that should be resolved

Type tree analysis results

Messages include:

Error number

L for error in logical analysis

S for error in structural analysis

Text message describing error

(error) or (warning)

Interpreting type tree analysis error messages

1. Identify type in error

2. Read error message relative to type in error

3.

Double click error message in Analyzer Results view to find and select type in type tree

4. Identify component in error by component number


6/53

Displaying component numbers


7/53

Validating data definitions

Using Map Designer

Create maps to specify logic necessary to transform data:

Identify data sources and targets

Identify type trees that describe format and structure of data sources and targets

Provide map rules that describe transformation requirements and business rules

Map Designer user interface

Map Designer basics

Fromwindow

Data sources (inputs)

Shows format of input data

Towindow

Data targets (outputs)

Shows format of output data

Cards represent data objects

One input card for each data source

One output card for each data target

Rule bar used to create and view complete map rules

Rule column displays portion of map rules


8/53

Map preferences

Maps and map source files

Mapis a definition of how to generate outputs from given inputs

Map source file(.mms file) stores one or more map definitions

Composition view

Shows map source file and map definitions they contain

Shows map relationships

Arranged in compositional hierarchy

Functional maps listed beneath executable map that calls it

New maps only appear when at least one card is defined


9/53

Outline view

Shows map definitions in alphabetical

order

In expanded view

Shows input cards and outputcards for the map definition

Provides access to map

Organizer views

New maps appear immediately in

Outlineview

Navigator map icons

Color and type of icon in the map

navigator provides general information

regarding the map purpose and content

Main, executable map definition

Functional map definition

Map Solution Diagram

Provides graphical view of map

relationships

1. Right-click map in map Outline

or Composition view2.

Select Map Solution Diagram


10/53

Defining an input card

At a minimum:

1.

Enter CardNamethat describes input

2.

Select TypeTreethat

contains input data definition3. Select Typein type tree

that defines input data

4. Specify Sourceresource adapter

5.

Enter adapter specific

settings

How card definition relates to card view

CardNameappears in card header and a first object in tree structure

Typeappears in parentheses next to card name in card header and determines top level

object for mapping


11/53

Defining an output card

At a minimum:

1.

Enter a descriptive CardName

2.

Select the type tree

that defines the output

data format

3.

Select Typefrom

the type tree that

defines output data

4. Select Targetadapter

5.

Specify adapterspecific settings

Map Designer card hierarchy

Executable map

Map that is built and run

Top level map that operates on entire files, tables and messages

Minimally must include data description (type tree) and physical location of data (source or

target settings) on cards


12/53

Creating an executable map

1. Open the Map Designer

2. Create a new map source file

or

Open an existing map source file

3. Create a new map definition (Map > New)

4.

Add an input card for each data source (Card > New)

5.

Add an output card for each data target (Card > New)

6. Enter map rules

7.

Build (compile) the map

How the Transformation Extender engine works

.Validates input data against type tree

starting with Input card #1

.Applies map rules starting with Output card #1


13/53

Data validation

Parses data from deepest level out:

1.

Validates each field (level 2) in a

record against type properties,

restrictions and component rules2. Validates record (level 1) against

group properties and

component rules

3.

Repeat steps 1 and 2 for each

record until end of file (level 0)

4. Validates file (level 0) against

group properties and

component rules

Validation map: method #1

Map contains input and output cards

Same TypeTreeand Typein both cards

Map creates an output file that can be compared to input file

If input file exactly matches output file, type tree correctly defines data



14/53

1. Contract tree in input card and output card to only show top level object

2. Map top level object in input to top level object in output to force validation of input data

against type tree definitions and create output file


Map contains input card only

No output file created

Eliminates rules processing

Preferred method when validating large data files

Modifying the type tree

Always open the type tree from the map to ensure that that the type tree is the same one

referenced in the map


15/53

After making changes to trees

Map uses the saved version of type tree

When the type tree is changed, follow this procedure:

Option to always save after analysis in Transformation Extender Preferences

Troubleshooting aids

Command Server confirmation window

Run Results views

Trace file

Map debugger

Audit logs

Data audit log

Execution audit log

Command Server confirmation window

Displays information that may be of value for debugging

Return message

Input and output object counts

Run time


16/53

Run Results views

1.

Select Map > View Run Resultsor click to verify that map:

Read correct input

Generated correct output

2.

Select inputs and outputs

to display

Trace file contents

Summary trace to determine card that failed

Trace on input (InputContentTrace)

Data objects found

Why data is found to be not valid

Sizes and counts of data objects

Position of data objects in data stream

Trace on output (RulesTrace)

Data objects built

Data objects that evaluated to NONE

When to use trace

Generate an input trace when map returns a data related error during validation

Generate an output trace when a step-by-step account of data object generation is required


17/53

Generating the trace file

1. Select Map

Settingsfrom

Mapmenu

2. Expand

MapTrace

options

3.

Set Switchto ON

4. Specify trace

options

5. Build map

6.

Run map

Viewing the trace file

Reading the trace file

Trace file is text file in the format mapname.mtr where mapnameis the name of the

executable map

Use Edit >Find/Replaceto search within trace information to locate error


18/53

Search on words such as Not, Failed, Wrong, or Invalid

Look immediately above or below error in trace file for additional information

Summary trace

1.

Generate a summary trace to identify the card in error

2.

If an error occurred on input during validation, run an input content trace on the card

If an error occurred on output, run map debugger


19/53

Interpreting input content trace results

Interpreting trace results

Reasons for validation failure

Input data did not match a type property in type tree

Data object has wrong Initiator or Terminator

Group is missing a required component

Delimiter does not match definition

Data object failed a component rule

Data object not in restriction list


20/53

Example: Input data did not match type property

Missing required component


21/53

Input Valid, Unknown Data Found

Transformation Extender found enough data to meet definition but found additional data

Usual causes

Source object defined as having a single occurrence

Source object defined as having zero or more objects and Transformation Extender

has found at least one, but it is followed by additional data

Functional map being built and run

Undefined data at end of file

Undefined data at end of file

Incorrectly defined data


22/53

Additional validation

During basic validation, data is isolated and compared to type definitions

Additional validation can be added to prevent processing of invalid data

Additional validation can be added to check if data, once isolated, matches specific datavalue or set of values

Use a restriction listto limit item to particular value or set of values

Use a component ruleto specify condition that must be met for a particular

component to be valid

Restriction lists

Limit valid values of item to a specific set

After changing restriction list, type tree must be analyzed and map re-built

Restriction list is case sensitive unless the Ignore caseitem property is set to Yes

Disabling restriction validation

Restrictions can be ignored when running a map

Set ValidationMap Setting to Custom

Set RestrictionErrorto ignore


23/53

Restriction settings

Include or exclude

Value restrictions

Range restrictions

Character restrictions

Include or exclude value restrictions

Included values are considered to be valid

Excluded values are considered to be not valid

Include or exclude values supported for:

Characters

Numbers

Dates


24/53

Identifying restriction values

Use when object data is limited to a set of static values

To add restrictions:

Double-click item type in Type tree view or

right-click and select Open

Enter values in restriction columns

Restriction view column content varies based on restriction property settings

Include character restrictions

Include First

Character list of valid first character Include After

Character list of valid characters that

may follow first character

To specify common range of characters,

enter reserved word in restriction list:


25/53

Exclude character restrictions

Exclude is list of character

substrings to be excluded

Reference Stringis character string that replaces excluded character substring on output

On input, any data that contains any substring in the Excludelist would be marked as invalid

On output, a character text item would be built by using the Reference Stringif the content

contained any of the Excludecharacter substrings

Include or exclude value ranges


26/53

Trace file with restriction error

Component rules

Used for data validation

Statement about component that determines if data is valid

Evaluate business rule

Evaluate to TRUE or FALSE

Rule can refer to:

Component it applies to

Any component earlier in same component list


27/53

Shorthand notation for rules

Use $as shorthand to represent current component in rule

Creating component rules

Created in rule bar

Must include full component name in rule

Press and hold Ctrlto drag and drop component name into rule bar

Always press Enter to save changes


28/53

Component rule references

Rule can refer to:

Component it applies to

Nested components of any component that can be referred to

Using functions in component rules

Component rules can use most Transformation Extender prebuilt functions


29/53

What is a function?

Expression that produces a result based on a certain operation

Can be nested with other functions

Can be entered in component rules and map rules

Not all Transformation Extender functions are available for use in component rules

Example: COUNT function

Returns number of occurrences of an object

Syntax: COUNT (series-object-expression)

Meaning: COUNT (objects_to_count)

Returns: Result is an integer equal to the number of objects_to_count.If input

argument evaluates to NONE, COUNT returns zero.


30/53

Functions view

Lists all Transformation Extender functions

Includes description of function syntax

Drag-and-drop

function from

Functions view

into component rule

Component rules

Component rules always evaluated even if an optional component is not present in data

Use WHEN(PRESENT ) if component rule should only be evaluated when data is present

Sample trace message


31/53

failed COMPONENT RULE test appears in trace message when component rule evaluates to

False

Comments in component rules

Comment can be added to component and map rules

-

Begin with /* and end with */

-

Appear anywhere in a rule

- Cannot be placed within object name


32/53

When to use what

For data validation use:

Restriction list

When an item is limited to a particular set of valid values

Component rule

To specify a condition that must be met for a particular component to be valid

For assigning a value to an output group or item, use a mapping rule


33/53

Map Designer methodology

Map rules

Describe how to build an output data object

Can be any combination of the following:

Hard-coded string enclosed in double quotes

Hard-coded numeric values

Map functions

Object names

Mathematical functions

Concatenation of multiple objects using a plus (+) character

Functional map references

Always start with an equal sign (=) character

Are required for every output object

Enter =None for objects that do not require output


34/53

Map rule examples


35/53

Functions

Text

BCDTOTEXT

COUNTSTRING

DATETOTEXTFIND

FILLLEFT

FILLRIGHT

HEXTEXTTOSTREAM

ISALPHA

ISLOWER

ISNUMBER

ISUPPER

LEAVEALPHA

LEAVEALPHANUM

LEAVENUM

LEAVEPRINT

LEFT

LOWERCASE

MAX

MID

MIN

NUMBERTOTEXT

OFFSET

PACKAGE

PROPERCASE

REVERSEBYTE

RIGHT

SERIESTOTEXT

SIZE

SQUEEZE

SUBSTITUTE

TEXT

TEXTTOBCD

TEXTTODATE

TEXTTONUMBER

TEXTTOTIME

TIMETOTEXT

TODATETIME

TONUMBER

TRIMLEFT

TRIMRIGHT

UPPERCASE

WORD

Conversion

BCDTOHEX

BCDTOINT

BCDTOTEXTCONVERT

DATETONUMBER

DATETOTEXT

FROMBASETEN

FROMDATETIME

FROMNUMBER

HEXTEXTTOSTREAM

INT

NUMBERTODATE

NUMBERTOTEXT

PACKPACKAGE

REFORMAT

SERIESTOTEXT

SYMBOL

TEXTTOBCD

TEXTTODATE

TEXTTONUMBER

TEXTTOTIME

TIMETOTEXT

TOBASETEN

TODATETIME

TONUMBER

UNPACK

UNZONE

ZONE

External Interface

DDEQUERY

DBQUERY

DBLOOKUP

ECHOIN

EXIT

HANDLEIN

GET

PUT

RUN

Lookup and Reference

CHOOSE

DBLOOKUP

DBQUERYDDEQUERY

EXTRACT

GETANDSET

GETDIRECTORY

GETFILENAME

GETPARTIONNAME

GETRESOURCENAME

INDEX

INDEXABS

LASTERRORCODE

LASTERRORMSGLOOKUP

MEMBER

SEARCHDOWN

SEARCHUP

SORTDOWN

SORTUP

UNIQUE

Bit Manipulation and

Testing

OFFSET

SETOFF

SETON

TESTOFF

TESTON

Error Handling

CONTAINSERRORS

FAIL

ISERROR

LASTERRORCODE

LASTERRORMSG

ONERROR

REJECT

VALID

Date and Time

ADDDAYS

CURRENTDATE

CURRENTTIMECURRENTDATETIME

DATETONUMBER

DATETOTEXT

FROMDATETIME

MAX

MIN

NUMBERTODATE

TEXTTODATE

TEXTTOTIME

TIMETOTEXT

TODATETIME

Math and Statistics

ABS

COUNT

FROMBASETEN

INT

MAX

MIN

MOD

ROUND

SQRT

SUM

TOBASETEN

TRUNCATE

Logical

ALL

OR

NOT

IF

EITHER

WHEN


36/53

What can map rules do?

Map input object to output object

Extract input records based on specific criteria

Count a conditional number of records

Map input records based on specific qualifier to specific output record

Convert an object in input data from one value to another value

Perform conditional logic

Run another map to produce an output

Entering map rules

Map rules entered in rule bar

Colors distinguish functions from arguments, object names and other map rule components

Key words can be configured with different colors

Invalid rule can be highlighted with background color

Always press ENTERto accept changes

Always drag-and-drop or copy and paste object names from cards

Getting ready for map rules


37/53

New card default is collapsed view

Expand group objects to lowest level

Grayed rule cells or objects occurring multiple times indicate possible functional

map use

Must enter a rule in every rule cell that is not gray

Mapping items

If output object is an item type, enter map rule at item level

Mapping groups occurring once

If output object is a group that occurs exactly once, enter map rule at group oritem level

(notboth)

Mapping groups in a series

If a group or an item can occur more than once, decide how many occurrences are to be

built

If there is a known number of outputs, use indexing


38/53

If the number of outputs depend upon a variable number of inputs, use afunctional

map

Indexing an output

Used when you want to:

Map a specific, known number of outputs one object at a time

Create test data

Indexing an output


39/53

Managing indexes

Reference a specific index by enclosing instance number in square brackets

Example: Record [2] to reference the second record number

Add an index

1. Right-click object

2. Select Add Index

Delete an index

1.

Right-click object

2.

Select Delete Index

Rearrange indexes

1. Select index object

2.

Press Ctrl+Shiftand drag to desired location

3.

Moves rules associated with selected object to new location


40/53

Functional maps

What is a functional map?

Like a subroutine

Maps portion of data at a time

Number of output objects are created produced based on some variable number of

corresponding input objects

Is not built and run separately from main map

Does not contain references to data sources and targets

Defining a functional map

1.

Determine need for a functional map

2.

Determine input arguments to functional map

3. Create functional map call

4. Create the functional map

Manually by creating a new map and input and output cards

Automatically using the Functional Map Wizard

5.

Enter map rules in functional map output card

Determining the need

Create a functional map when:

Output group object can occur more than once (component range is greater than 1)

and

Some unknown number of objects are created in output (component range is s)

and

Output group type is different from input group type

Determining the arguments

Functional map syntax is used to:

Name the functional map


41/53

Specify data to pass to the functional map

Functional maps only have access to data passed to them through function arguments

Identify which objects are required to produce output

Data objects must be passed from calling map as an argument in the functional map

call

One argument is thought of as a trigger

I want one output object for each ...

Creating the functional map call

Created on a component that occurs an unknown number of times (range of s)

At least one argument should be component that occurs an unknown number of times

Recommended to precede functional map name with F_to easily distinguish functional

maps

Creating the functional map call example

Functional map example


42/53

One input card created for each argument in functional map call

Order of input cards matches order of arguments

One output card always created

Creating functional map manually

1. Create a new map (Map > New)

2. Enter new map name exactly as in functional map reference

3.

Add one input card for each argument in the functional map call (Card > New)

4.

Add one output card (Card > New)

5. Enter map rules in the functional map

Creating a functional map using the Functional Map Wizard

Automatically creates functional maps based on map rules

Creates input and output cards

Creates multiple maps when a rule contains calls to more than one functional map


43/53

Functional Map Wizard results

Functional map is created in the current map source file (.mms) using the parameters

specified in the Functional Map Wizard dialog


44/53

Functional Map Wizard icons

Meaning Action to Take

Card is undefined (card name, type

tree file, or type name is not valid)

In the Functional Map Wizard dialog, specify

a valid card name, type tree file, or type

name.

Invalid input argument in a

map rule.

Exit from the Functional Map Wizard and

change the invalid input argument in the

rule.

Everything has been properly

specified for given card or map

No action necessary. Functional map can be

created.

Modifying card attributes


45/53

Functional Map Wizard restrictions

Does not automatically create input cards for arguments defined as expressions (such as

literals) that evaluate to text or number items

Does not run if an argument is invalid

Does not dynamically update if changes are made to the functional map call

Main and functional map input cards

Functional map output cards


46/53

From arguments to cards

Functional map input cards

One input card must be created for each argument passed from calling map

Input cards must be created in same order as arguments


47/53

Functional map output card

Only one output card per functional map

Type matches type of output in calling rule

Uses same type tree as output card in calling map

Argument must match input card


48/53

Card must match output type

Evaluating functional maps

Evaluated once for each combination of occurrences of each argument passed

To conditionally control input to a functional map or output of an object, use functions and

expressions within a functional map call


49/53

Example

=F_MakeNewOrderRecord (EXTRACT(LegacyRecord:LegacyFile, PARTITION ( ShipToCode

Field:.:LegacyFile, West)), XrefFile )

If any argument evaluates to NONE, the functional map will not be called for thatcombination of arguments

Functional maps build output; functions do not

Tracking rules

Rules are saved with output name, not position

If output objects change sequence, rules are maintained

If object disappears or the name changes, rules become unresolved

What causes unresolved rules

- Name of output type changes in type tree

- Output card is changed to point to a different object

- An intermediate level is added between object in output

Displaying Unresolved Rules view

Automatically displayed when unresolved rules are present

Can be selected from Show Viewmenu


50/53

Resolving unresolved rules

Drag rules from unresolved rules into place

Use Modifyto find and replace changed type names

Must delete all unresolved rules before building map


51/53

Map Debug option

Debug complex rules step by step in Map Designer

Set breakpoints on any rule, in executable or functional maps

Visually step into every rule in every map

Inspect data while it is changing

Step In, Step Out, Step Over, and Run commands

Debug maps remotely by referencing map source locally

Map Debugger features

Enable the Debug Mapiconto switch Map Designer into debug mode

When enabled, allows setting of break points on output objects with rules

Building maps with debug mode enabled produces separate map debug file

No changes to compiled map (.mmc) structure, so maps built with debug are same as other

compiled maps

Map Debugger usage


52/53

Map Debugger operation

Enable debug and build map

Set breakpoints

Run map

Debug mode causes Debug view window to appear

When breakpoint is reached, engine pauses and the rule and its composite functions

and objects appear in Debug view window

Use buttons in Debug view window to continue processing

Adding breakpoints (1 of 2)


53/53

Debugger view window

Map Debug options

Step Into see what data is held by objects referenced in rule and data returned by functions

and functional maps

Steps into local, non-referenced functional maps and RUN maps contained in same

map source file (.mms)

Literals and operators are not shown

Step Thrugoes to end of current rule

Does not show resultant object values or function returns

Step Overignores current iteration of a rule and goes to beginning of the next rule to be run

Step Thru/Overprocesses rules in functional maps or RUN maps called from the current

rule, unless a breakpoint is set

defining components

Documents