cooperating system release notes

Co>Operating SystemRelease Notes 2.13

May 2006 > Part Number AB0630

Ab Initio Software Corporation201 Spring Street > Lexington MA 02421 > Voice 781.301.2000 > Fax 781.301.2001 > [email protected]

NOTICE

This document contains confidential and proprietary information of Ab Initio Software Corporation.Use and disclosure are restricted by license and/or non-disclosure agreements. You may not access,read, and/or copy this document unless you (directly or through your employer) are obligated to AbInitio to maintain its confidentiality and to use it only as authorized by Ab Initio. You may not copythe printed version of this document, or transmit this document to any recipient unless the recipientis obligated to Ab Initio to maintain its confidentiality and to use it only as authorized by Ab Initio.

Copyright Ab Initio Software Corporation. All rights Reserved : DOC# AB0630

INTELLECTUAL PROPERTY RIGHTS & WARRANTY DISCLAIMER

CONFIDENTIAL & PROPRIETARYThis document is confidential and a trade secret of Ab Initio Software Corporation. This document is furnished under a licenseand may be used only in accordance with the terms of that license and with the inclusion of the copyright notice set forth below.

COPYRIGHTSCopyright 1997 - 2006 Ab Initio Software Corporation. All rights reserved.

Reproduction, adaptation, or translation without prior written permission is prohibited, except as allowed under copyright law orlicense from Ab Initio Software Corporation.

TRADEMARKSThe following are worldwide trademarks or service marks of Ab Initio Software Corporation (those marked are registered inthe U.S. Trademark Office, and may be registered in other countries):

Certain product, service, or company designations for companies other than Ab Initio Software Corporation are mentioned inthis document for identification purposes only. Such designations are often claimed as trademarks or service marks. Ininstances where Ab Initio Software Corporation is aware of a claim, the designation appears in initial capital or all capitalletters. However, readers should contact the appropriate companies for more complete information regarding such designationsand their registration status.

RESTRICTED RIGHTS LEGENDIf any Ab Initio software or documentation is acquired by or on behalf of the United States of America, its agencies and/orinstrumentalities (the Government), the Government agrees that such software or documentation is provided with RestrictedRights, and is commercial computer software or commercial computer software documentation. Use, duplication, ordisclosure by the Government is subject to restrictions as set forth in the Rights in Technical Data and Computer Softwareprovisions at DFARS 252.227-7013(c)(1)(ii) or the Commercial Computer Software Restricted Rights provisions at 48 CFR52.227-19, as applicable. Manufacturer is Ab Initio Software Corporation, 201 Spring Street, Lexington, MA 02421.

WARRANTY DISCLAIMER THIS DOCUMENT IS SUBJECT TO CHANGE WITHOUT NOTICE. AB INITIO MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIAL,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. AB INITIO SHALLNOT BE LIABLE FOR ERRORS CONTAINED HEREIN OR FOR INCIDENTAL OR CONSEQUENTIAL DAMAGE IN CONNECTION WITH THE FURNISHING,PERFORMANCE, OR USE OF THIS MATERIAL.

>Ab Initio I>OAb InitioABINITIO.COMCo>Operating EnterpriseCo>Operating SystemCo>OperatingCo>OperationCo>OperativeCo>OpSys

Continuous FlowsContinuous>FlowsCooperating EnterpriseCooperating SystemCooperatingData>ProfilerDynamic Data MartEMEEME PortalEnterprise Management Environment

Enterprise Meta>EnvironmentEnterprise Metadata EnvironmentEnterprise MetaEnvironmentEverything 2 EverythingEverything to EverythingFrom The BeginningI>OIf You Can Think It, You Can Do Itinit.comINIT

Meta Operating SystemMeta OSMeta>Operating SystemMeta>OSRe>PositRe>SourceServer++Server+ServerShop for DataThe Company Operating System


Contents iii

Contents AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY

ABOUT THIS BOOK IXSubject ixAudience ixAb Initio documentation ix

PDFs and printed books ixRelease Notes ixBasic book set ixOther documentation x

Documentation conventions xConventions for text xConventions for examples xiConventions for syntax xi

Location of more information xiNew users xiGDE users xiiApplication developers xiiSystems administrators xii

ABOUT THIS RELEASE 1GDE requirement 1

Supported platforms 2

Key files 3Moving a key file between computers 3

Windows Korn Shell Requirement 5Installation 5Operation 5New rexec service 5

Secure connections with Cygwin and pdksh 6Connection capabilities 6Keys 6passwd and group files 6File permissions 6Authentication 6


DO NOT COPY

Setup instructions 7

Location of more information 11iv Contents AB INITIO CONFIDENTIAL AND PROPRIETARY

New users 11GDE users 11Application developers 11Systems administrators 11Installation at a Windows computer 11Installation at a UNIX computer 12

Location of online examples 13GDE examples 13SDE examples 13

Ab Initio Environment 14

Linux platform compiler change 15

Flags for newer compilers 16

Connection methods by platform 17

Upgrade guide 18Interoperability with earlier versions 18Upgrade the GDE 18Install the new Co>Operating System release 19Set AB_COMPATIBILITY 20Test 20Compatibility issues 20Assignments between vectors of different size 26

NEW FEATURES IN RELEASE 2.13 29New DML functions 30

New functions 30New AB_CHECK_BINARY_INTEGER_CONVERSION configuration variable 30New reinterpret( ) function 30New built-in functions string_to_hex and string_from_hex 31New built-in functions scanf_double and scanf_long 32New built-in function string_representation 33New date functions 33date_add_months 33date_difference_days 34


Contents v

date_difference_months 34datetime_add_months 35AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY

datetime_difference_days 35datetime_difference_hours 36

Details 36datetime_difference_minutes 36datetime_difference_months 37datetime_difference_seconds 37New DML eval function 37

Other new DML features and changes 39Validation now done on date assignments 39Unsigned zoned decimals 39New V specifier in delimited dates and datetimes 39Length-Prefixed Vectors 39Key-based operations extended to more types 41Assignment to union members 41NULLs in control flow statements 43Changes in ab_type_info utility 43Escapes in U"..." and other string constants 44cobol-to-dml 44Named constants in DML 44

Co>Operating System changes and new features 46Java multiplexer 46Remote queue manager for MQ Publish components 46New AB_COMMIT_PROGRESS_NARRATION configuration variable 47

AB_TIMEOUT configuration variable 47New Continuous Flows recovery mechanism 47

Contents of the reject capture directory 47Example of use 48

AB_STARTUP_PASS_ARGS_LATE configuration variable 49New flag for m_queue utility 49New parameter for Publish and Multipublish components 49New allocation option keyword for MVS Input File 49m_queue utility enhancements 49Compressed flows 50Data encryption for flows 51New ab-cat-npipe utility 51


DO NOT COPY

Rlogin connection method eliminated 51Support for very large files 52vi Contents AB INITIO CONFIDENTIAL AND PROPRIETARY

New component features 54Read Multiple Files can repair malformed records 54RunCommands component 54Enhancements in SAS components 55New parameter for JMS Publish and JMS Subscribe 55New commit_level parameter for JMS and MQ components 55New Send Mail component 56New Call Web Service component 57New LDAP Reader component 57New parameter for Read Raw component 57XML Schema support and the Full Object Model 57Other XML enhancements 64Read Tagged Values component changes 64Denormalize component restriction lifted 65New components: Inflate and Deflate 65Output File contents can be directed to stdout or stderr 65New poll_delay parameter in MQ Subscribe 66New component Scan with Rollup 66Run Program component accepts single quotes 67FTP component password 67New parameters for FTP To and FTP From components 68Log port for FTP components 69Special files as component output 69

Platform-specific features 70Windows: Client-side port ranges can now be specified by abexecserv DCOM server 70z/OS: AB_ZOS_COMPONENT_JOBNAME configuration variable 70

Database package new features 71

Netezza 3.0 72

Sybase error handling 73New configuration variables 73New runtime error policies 73

DB2 exception tables 75Exception table name 75


Contents vii

Package name 76

DB2 load parameters 77AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY

DB2 load log files 78

Other database package new features and information 79Output Table utility mode and MS SQL Server 79Multiple Netezza db_nodes values 79New Output Table behavior in db2uss utility mode 79New basic_odbc tag 80DB2 CLI database aliases 80New copy_target parameter for DB2 Output Table 80Sybase IQ utility-mode Output Table 80DB2 DEC and INT abbreviations recognized 81New db2_force_remote DBC file tag 81Inline .dbc file comments 81New Netezza loader 82DATE and TIMESTAMP columns in the same Oracle table 82Unloading Oracle CHAR columns with character length semantics 83Stored procedures and m_db run and Run SQL 83Calling Oracle stored procedures through synonyms 83Overloading of Oracle stored procedures 83AB_IDB_CLIENT_VERSION configuration variable 84IDMS Input Table 84Excel support 84Enhancement to Output Table used with Netezza 84right and left now allowed in SQL 84IBM AS/400 suport 84New Join with IMS component 84New Join with IMS Segments component 84Support for Informix 85Utility mode for Input Table for DB2 for OS/390 85Generated DML 85Sybase optional .dbc file fields for maximum datatype sizes 85do_query( ) transform function for Join with DB 86Teradata Continuous Update Table 86DB2 reject files nomenclature change 86New .dbc file tag session_startup_sql 87


DO NOT COPY

New database configuration tag for ODBC connections 88Transform package for Multi Update Table 88viii Contents AB INITIO CONFIDENTIAL AND PROPRIETARY

Deadlocks handled in Multi Update Table components 88Oracle 7 and 8.0.x support 88Parallelizing Oracle queries 89Record delimiters for Output Table with Oracle 90

DATABASES SUPPORTED BY PLATFORM 91Databases supported by the Ab Initio Database Package 92

Databases supported on AIX 93

Databases supported on Digital/Compaq Alpha-Tru64 (OSF1) 95

Databases supported on HP-UX 96

Databases supported on NCR MP-RAS 97

Databases supported on OS/390 (z/OS) 98

Databases supported on Linux 99

Databases supported on Solaris 100

Databases supported on Windows NT/2000 102


ix

About this bookAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY

SUBJECT Co>Operating System version 2.13 release information.

AUDIENCE This manual is aimed at developers and administrators who are installing and/or using the Ab Initio Co>Operating System.

AB INITIODOCUMENTATION

Ab Initio provides documentation in online help, in Adobe PDFs, and in print.

PDFs and printedbooks

If you do not have the PDFs or printed books you need, contact your Ab Initio account representative.

Release Notes z To view Release Notes either for the Co>Operating System that is your current run host or for the GDE, select Help > Release Notes from the GDE menu bar.

z To view Release Notes for Data Profiler, select Help > Release Notes from the Data Profiler menu bar.

z To obtain Release Notes for other Ab Initio products, contact your Ab Initio account representative.

Basic book set The following is the basic set of Ab Initio books:

SUBJECT BOOK

System management Co>Operating System Administrators Guide

The core components Component Reference

Connecting to and using databases Database Package Guide and Reference

Data Manipulation Language (DML) Data Manipulation Language Reference

DML built-in functions Data Manipulation Language Function Reference

Graphical Development Environment (GDE) The Tutorial


OT COPY

SUBJECT BOOKx AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Otherdocumentation

Other Ab Initio products and their corresponding documentation are the following:

See also Location of more information on page xi.

DOCUMENTATIONCONVENTIONS

Ab Initio documentation both online and in print uses font formatting and symbols to convey special meaning of particular elements in text, examples, and syntax.

Conventionsfor text

Graph programming Shell Development Environment Users Guide

Latest information on the Co>Operating System Release Notes

PRODUCT DOCUMENT

Continuous Flows Guide to Continuous Flows

Data Profiler Data Profiler Guide and Reference

Release Notes

Enterprise Meta>Environment (EME) Guide to Managing Technical MetadataGuide to Managing Enterprise Metadata

EME Reference

Release Notes

Shop for Data (SFD) Shop for Data Administrators Guide and ReferenceShop for Data Developers Reference

Shop for Data Developers Guide

Release Notes

EXAMPLE DESCRIPTION

mainfile.txt Bold represents user-entered values, file paths, DML keywords, numerical or character values (such as abc or 1), as well as the names of fields, functions, statements, and parameters.

File > Open Bold italic represents text that appears in the user interface. For example: Choose the Open command from the File menu.

AB_HOME Small caps represent names of configuration, environment, and system variables and names of components.

my_file Italic represents variables as well as emphasis, new terms, and book titles.


xi

Conventionsfor examples

EXAMPLE DESCRIPTIONAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY

Conventionsfor syntax

LOCATION OFMORE

INFORMATION

The following documentation contains information about Ab Initio software.

New users The following contains material of interest to new users:z Graphical Development Environment (GDE) Online Tutorial (Select Help > Tutorial.)

z Tutorial (In hardcopy)

z The GDE online Help (Select Help > Contents.)

source code A fixed-width font represents code fragments, examples, and computer output.

user input A bold fixed-width font represents entries made by the user in an interactive command-line session.

An arrow indicates the result of a computation. Squares indicate blank spaces where the number of spaces is significant.

EXAMPLE DESCRIPTION

integer Non-bold text indicates the DML data type of an argument or returned element.

read_xml Bold text, symbols, and punctuation represent syntactical elements that you must enter exactly as they are shown.

my_file Italicized text represents values or variables that you must supply.

... An ellipsis indicates that the preceding item can be repeated one or more times.

a | b | c The logical or symbol separates alternatives.

( ) Bold parentheses are part of the syntax.

[ ]

[ ]

Non-bold square brackets surround one optional item or a series of optional items. If optional items are separated by the logical or symbol, you may choose one or none. If the or symbol is not present, you may choose as many as you want or none.

Bold square brackets are part of the syntax.

{ }

{ }

Non-bold curly braces indicate a series of choices from which you must select one.

Bold curly braces are part of the syntax.


OT COPY

GDE users All GDE users will find the following of interest:z GDE Release Notes (Select Help > Release Notes.)xii AB INITIO CONFIDENTIAL AND PROPRIETARY DO N


Applicationdevelopers

The following documentation is intended mainly for application developers:


z Shell Development Environment (SDE) Users Guide

z Data Manipulation Language (DML) Reference

z Working with Components

z Component Reference Volume 1 and Volume 2

z Database Package Guide and Reference

z Co>Operating System Administrators Guide

Systemsadministrators

Application developers, system administrators, project leaders and operators should also look at the Co>Operating System Administrators Guide.


ase 1AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this rele

1>

About this release The Co>Operating System 2.13 Release Notes are addressed particularly to readers interested in Release 2.13, or in upgrading from Release 2.12.2 to Release 2.13 of the Co>Operating System. If you are interested in upgrading from a pre-2.12.2 release of the Co>Operating System to Release 2.13, you should refer to the document Ab Initio Co>Operating System Release Notes 2.13: Supplement for Upgrading from Release 2.11.7 and earlier releases).

z Supported platforms (page 2)

z Key files (page 3)

z Windows Korn Shell Requirement (page 5)

z Secure connections with Cygwin and pdksh (page 6)

z Location of more information (page 11)

z Location of online examples (page 13)

z Ab Initio Environment (page 14)

z Linux platform compiler change (page 15)

z Flags for newer compilers (page 16)

z Connection methods by platform (page 17)

z Upgrade guide (page 18)

GDE REQUIREMENT Note that in order to use the Release 2.13 features of the Co>Operating System, you must have installed Release 1.13 of the Graphical Development Environment (GDE).


OT COPY

Supported platforms 2 About this release > Supported platforms AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

This release of the Ab Initio Co>Operating System supports the following platforms. (The list of platforms supported by the Co>Operating System is not necessarily identical to the list of platforms supported by other Ab Initio products.)

z Sun Solaris 2.6, 7, 8 and 9 (SPARC)

z IBM AIX 4.3.3, 5.1, 5.2, and 5.3

z Hewlett-Packard HP-UX 11 and 11i (PA RISC)

z Silicon Graphics IRIX 6.5

z Linux (x86)

The following Linux distributions are supported on x86:

z Red Hat Enterprise Linux AS versions 2.1 and 3.0

z SuSE Linux Enterprise Server 8

z Windows NT 4.0 (x86) with Service Pack 6a or later

z Windows 2000, Windows XP and Windows 2003 (x86)

z Compaq Tru64 UNIX Version 5.1 or later

z NCR MP-RAS 3.02

z IBM OS/390 Versions 2.9 and 2.10

Note both OS/390 and z/OS are referred to generically as z/OS in the rest of this document.

z IBM z/OS 1.1, 1.2, 1.3, and 1.4

For information about the using the Co>Operating System on IBM OS/390 platforms, see IBM OS/390 information in Chapter 5, Addendum, of the Co>Operating System Administrators Guide.


files 3

Key files AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Key

The Co>Operating System, like other Ab Initio products, requires a valid key file to run.

On startup, the Co>Operating System looks for a key file in $AB_HOME/config. Key files are tied to specific computers, and so cannot be copied from one computer to another.

If you have not yet received a key file for your computer:

1. Run the utility $AB_HOME/bin/ab-computer-info.

This program prints information about your computer, which Ab Initio needs to create a key file for you.

2. Email the output from ab-computer-info to a designated Ab Initio contact in your company. If you don't know who that is, see your system administrator.

Your Ab Initio contact can use the information to send you a key file for your computer. Designated contacts can call 1-781-301-2588 or email [email protected].

To install a key file that you have received:

1. Place the key file in a directory that is accessible from the target computer.

2. Run the following command on the target computer:

$AB_HOME/bin/ab-key add pathname

The pathname can be either the key file itself or the directory containing the key file.

The ab-key utility finds all Co>Operating System and GDE installations and installs the key file into the appropriate location in each installation. If there are installations with existing key files, ab-key will prompt you to overwrite them.

MOVING A KEY FILEBETWEEN

COMPUTERS

To move a key file between computers, you must first remove it from the old computer, and then request a key file for the new computer.

1. Run the following command to remove a key file from the old computer.

$AB_HOME/bin/ab-key remove

The command:

z Finds all key files installed on your computer and offers to remove them.

On Windows platforms, the removal code file is placed in the shared installation directory, which is usually called:

c:\Program Files\Common Files\Ab Initio


OT COPY

The file itself is named

ab-remove-YYYY-MM-DD-##.txt4 About this release > Key files AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

where YYYY-MM-DD represent the year, month, and day when the key was removed. If the file already exists, ## represents a two-digit increment which is added to the name.

On other platforms, the removal code file is placed in your temporary directory and named

ab-remove-YYYY-MM-DD-##.txt

where YYYY-MM-DD represent the year, month, and day when the key was removed. If the file already exists, ## represents a two-digit increment which is added to the name.

z Creates a file containing the key removal information.

2. Send the file created by ab-key to a designated Ab Initio contact in your company to register the key removal.

NOTE: Failure to send this file to your designated contact prevents your key from being transferred to

another computer.

Once you remove the key file from the computer, you must request a new key file to be able to run the product on that computer again.


ent 5

Windows Korn Shell RequirementAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Windows Korn Shell Requirem

Under Microsoft Windows, the Co>Operating System requires a Korn Shell (a command-line interpreter and scripting language) and a number of Unix-like utilities. For previous versions of the Co>Operating System, installation of the MKS Toolkit was necessary to meet this requirement.

Beginning with Release 2.12 of the Co>Operating System, an alternative Korn Shell is also supported: it is called pdksh (Public Domain Korn Shell), and it runs as part of the Cygwin utilities.

INSTALLATION As of Release 2.12, pdksh and the Cygwin utilities are automatically installed as part of the Co>Operating System installation. If the MKS Toolkit is already present on your machine, pdksh and Cygwin are not installed.

Ab Initio has made a number of enhancements to pdksh and the Cygwin utilities. If you already have Cygwin installed on your machine and do not wish to overwrite it, you must contact Ab Initio support to obtain information about which versions of Cygwin are supported and to obtain the enhanced pdksh and Cygwin utilities.

OPERATION After installation, the Co>Operating System works automatically with the installed pdksh and the Cygwin utilities. If you wish to start a pdksh command-prompt window, an Ab Initio ksh shortcut is provided as a convenience under Programs in the Start menu.

Alternatively, you can type ksh -l -i at a command prompt or in the Run dialog. To learn more about pdksh, type man ksh at a prompt or in the Run dialog (then, when looking at the ksh manual, press u or d to scroll up or down, q to quit, or h for help).

NEW rexecSERVICE

A consequence of not installing the MKS Toolkit is that you cannot use the rexec service provided with it. However, beginning in Release 2.12, an rexec service is provided by Ab Initio with the Co>Operating System. Before you can use the service, you must start it, by executing the following command from a Korn shell:

XXnt-serv-mgr -rexecd -create -start


OT COPY

Secure connections with Cygwin and pdksh6 About this release > Secure connections with Cygwin and pdksh AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

If you are using Cygwin and pdksh (the Public Domain Korn Shell), you can use the OpenSsh package for making secure connections. The OpenSsh package imposes no limit on the number of connections you can have.

CONNECTIONCAPABILITIES

You can connect via ssh hetero from elsewhere (from a graph running on a different machine, for example) to Cygwin with or without a passphrase or password.

You can connect via ssh hetero from Cygwin to elsewhere only if you use a key pair with no passphrase. Using a keypair with a passphrase, or using no keypair (but with password authentication) will not work.

You can connect via ssh from the GDE to Cygwin with a key pair either using no passphrase, or using a password. You cannot connect from the GDE to elsewhere using a key pair with a passphrase (generated by the GDE or from elsewhere) in the current release of the Co>Operating System.

KEYS GDE-generated RSA2 and DSA2 keys work with Cygwin. Cygwin-generated RSA2 and DSA2 keys do not work with the GDE.

PASSWD ANDGROUP FILES

A blank /etc/passwd and /etc/group are shipped in Ab Initios Cygwin distribution. These must be populated for ssh to work correctly (see instructions below). However, doing this only makes a static snapshot of the system and the Domain. If the Domain changes (when a new user is added, for example), these files must be regenerated.

Note that having a blank /etc/passwd and /etc/group may result in Cygwin itself not behaving as you would expect it to. No normal Cygwin installation leaves these files blank.

FILE PERMISSIONS The Ab Initio installer does not set any file permissions: everything is installed inheriting the parent directorys permissions.

AUTHENTICATION Because of the way network authentication tokens are created, a login session to the Cygwin SSH server can only see network shares if you use password authentication. Using a key pair, you will be unable to mount or access network shares.


ksh 7

SETUPINSTRUCTIONS

1. Install the Co>Operating System with Cygwin and OpenSsh, or install OpenSsh from the Cygwin site.AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Secure connections with Cygwin and pd

2. Populate the passwd and group files with a static snapshot of (first) the local system and then the domain:

mkpasswd -l -g > /etc/passwdmkpasswd -d >> /etc/passwdmkgroup -l > /etc/groupmkgroup -d >> /etc/group


OT COPY

3. Configure ssh for the system. Following is an example of doing this:

ssh-host-config8 About this release > Secure connections with Cygwin and pdksh AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Generating /etc/ssh_host_keyGenerating /etc/ssh_host_rsa_keyGenerating /etc/ssh_host_dsa_keyGenerating /etc/ssh_config filePrivilege separation is set to yes by default sinceOpenSSH 3.3.However, this requires a non-privileged account called 'sshd'.For more info on privilege separation read

/usr/doc/openssh/README.privsep.

Shall privilege separation be used? (yes/no) noGenerating /etc/sshd_config file

Do you want to install sshd as service?(Say "no" if it's already installed as service) (yes/no) yes

You appear to be running Windows 2003 Server or later. On 2003 and later systems, it's not possible to use the LocalSystem account if sshd should allow passwordless logon (e. g. public key authentication). If you want to enable that functionality, it's required to create a new account 'sshd_server' with special privileges, which is then used to run the sshd service under.

Should this script create a new local account 'sshd_server' which has the required privileges? (yes/no) yes

Please enter a password for new user 'sshd_server'. Please be sure that this password matches the password rules given on your system. Entering no password will exit the configuration. PASSWORD=YourPasswordWARNING: Adding user sshd_server to local group root

User 'sshd_server' has been created with password 'YourPassword'.If you change the password, please keep in mind to change the password for the sshd service, too.

Also keep in mind that the user sshd_server needs read permissions on all users' .ssh/authorized_keys file to allow public key authentication for these users!. (Re-)running ssh-user-config for each user will set the required permissions correctly.

Which value should the environment variable CYGWIN have whensshd starts? It's recommended to set at least "ntsec" to beable to change user context without password.Default is "binmode ntsec tty".CYGWIN=ntsec

The service has been installed under LocalSystem account.


ksh 9

Host configuration finished. Have fun!AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Secure connections with Cygwin and pd

NOTE: If on Windows 2003 you wish to have a logon without password, you should select the option to

create an sshd_server account for the sshd service. You must also make sure that this account has

read permission on all users .ssh/authorized_keys files.

4. Now start the sshd service.

cygrunsrv --start sshdIf you get an Access denied error, verify that the local system user has access to execute the sshd program, which should be at

/usr/sbin/sshd

from pdksh or bash, or

C:\cygwin\usr\sbin\sshd.exe

in Windows Explorer.

5. Create your home directory as specified in the /etc/passwd file. This will be where the .ssh subdirectories go. It may already exist.

grep myusername /etc/passwd | awk -F ":" '{print $6}'/home/myusernanemkdir -p /home/myusernameAt this point you have two options. These are described in the two following steps.

6. If you are already using ssh elsewhere, just bring your private and public key into ~/.ssh and add your public RSA1 key to ~/.ssh/authorized_keys, or your public RSA2 or DSA2 key to ~/.ssh/authorized_keys2.

Configuration is the same as on Unix: ~/.ssh/config is the per-user configuration file, and /etc/ssh_config and /etc/sshd_config are the system-wide configurations for ssh and sshd, respectively. Note that if you change /etc/sshd_config, you will need to restart the sshd service:

cygrunsrv --stop sshd && cygrunsrv --start sshd

6. Alternatively (if you did not choose the other Step 6 above), you can generate new keys.

Keep in mind that only the RSA protocol 1 key will work with the GDE. If you want to use protocol 2 RSA or DSA keys, you must generate them in the GDE and then put their public sides into your ~/.ssh/authorized_keys2 file. Keep in mind that if you want a Cygwin Co>Operating System to be able to run hetero and ssh to somewhere else, you cannot use a passphrase.


OT COPY

To generate new keys, run ssh-user-config. Following is an example:

ssh-user-config10 About this release > Secure connections with Cygwin and pdksh AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Shall I create an SSH1 RSA identity file for you? (yes/no) yesGenerating /home/myusername/.ssh/identityEnter passphrase (empty for no passphrase):Enter same passphrase again:Do you want to use this identity to login to this machine?

(yes/no) yesAdding to /home/myusername/.ssh/authorized_keysShall I create an SSH2 RSA identity file for you?

(yes/no) yesGenerating /home/myusername/.ssh/id_rsaEnter passphrase (empty for no passphrase):Enter same passphrase again:Do you want to use this identity to login to this machine?

(yes/no) yesAdding to /home/myusername/.ssh/authorized_keys2Shall I create an SSH2 DSA identity file for you?

(yes/no) yesGenerating /home/myusername/.ssh/id_dsaEnter passphrase (empty for no passphrase):Enter same passphrase again:Do you want to use this identity to login to this machine?

(yes/no) yesAdding to /home/myusername/.ssh/authorized_keys2

Configuration finished. Have fun!

7. You can now copy the private or public keys to other machines, or import the RSA1 ~/.ssh/identity private key into the GDE.

8. At this point, you have completed the setup.


ion 11

Location of more information AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Location of more informat

The following documentation contains information about Ab Initio software.

NEW USERS For new users, see z Graphical Development Environment (GDE) Online Tutorial (Select Help > Tutorial.)

z Tutorial (In hardcopy)


GDE USERS For GDE users, see z GDE Release Notes (Select Help > Release Notes.)


APPLICATIONDEVELOPERS

For application developers, see:


z Shell Development Environment (SDE) Users Guide

z Data Manipulation Language (DML) Reference

z Working with Components

z Component Reference Volume 1 and Volume 2

z Database Package Guide and Reference

z Co>Operating System Administrators Guide

SYSTEMSADMINISTRATORS

For application developers, system administrators, project leaders, and operators, see the Co>Operating System Administrators Guide.

INSTALLATION AT AWINDOWS

COMPUTER

If you are installing the Co>Operating System at a Windows computer:

1. Insert the installation CD-ROM.

2. Click View/Print Install Guide.

One of the following appears depending on whether you have inserted a Windows or UNIX installation CD-ROM:


OT COPY

z Installation Guide for Co>Operating System and Database Package on Windows

z Installation Guide for Co>Operating System and Database Package on UNIX 12 About this release > Location of more information AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

3. Follow the instructions in the Installation Guide.

INSTALLATION AT AUNIX COMPUTER

If you are installing the Co>Operating System at a UNIX computer, see the hardcopy document Installation Guide for the Co>Operating System and Database Package on UNIX you received with the installation CD-ROM.


les 13

Location of online examples AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Location of online examp

Ab Initio provides examples accessible from the GDE and SDE.

GDE EXAMPLES To find the examples in the GDE, select Help > Examples.

SDE EXAMPLES Examples can be found in the following directories: z Examples of mp shell scripts:

$AB_HOME/examples/mp z Examples of transform applications:

$AB_HOME/examples/apps z Examples with the C++ API, including user-defined extensions to DML:

$AB_HOME/examples/c++ z Examples with the Ab Initio Data Manipulation Language:

$AB_HOME/examples/dml z Examples of XML datasets and graphs that operate on them:

$AB_HOME/examples/xmlTo use the XML examples, copy all the files in this directory to a directory of your own. Then, in the GDE, set up a Host Profile called XML Examples that points to your directory.

Note that $AB_HOME is the value of the environment variable AB_HOME, which should be set to the path to the Ab Initio installation at your site.


OT COPY

Ab Initio Environment14 About this release > Ab Initio Environment AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

The Ab Initio Environment is not included with the Co>Operating System. If you wish to obtain the Ab Initio Environment, please contact your Ab Initio account representative.


ge 15

Linux platform compiler changeAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Linux platform compiler chan

With the present release, the version of the GCC compiler used to build the Co>Operating System for Linux platforms is changed. The version of GCC previously used was 2.95.3. The GCC compiler version now used to build the Co>Operating System for Linux platforms is version 3.2.3.

This change affects, for example, customers who:

z build custom C++ components against the Co>Operating System API

z use DML extensions


OT COPY

Flags for newer compilers16 About this release > Flags for newer compilers AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

If you build custom C++ components against the Co>Operating System API, or if you use DML extensions, you will need the following information in order to use the newer compilers available on various platforms.

z z/OS

If you are using version 1.1 or a later version of z/OS, make the following setting in the shell environment in which you are doing the build:

export _CXX_CVERSION="0x220a0000"export _CXX_CLASSVERSION="0x220a0000"

z Solaris

If you are using version 5 or a later version of the Sun compiler, make the following setting in the shell environment in which you are doing the build:

export CXX_EXTRA_FLAGS='-compat=4'

z AIX

If you are using the Visual Age C++ compiler (version 6 or a later version), make the following setting in the shell environment in which you are doing the build:

export CXX_EXTRA_FLAGS='-qnamemangling=v5'


rm 17

Connection methods by platform AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Connection methods by platfo

The following table indicates the connection methods available on the supported operating systems. A check mark (9) indicates that the Co>Operating System supports the connection. A dash () indicates that it does not. The default connection method is rsh.

On the following operating systems...

The Co>Operating System supports the following connection methods as indicated...

rsh rexec ssh telnet1

1. Use rsh, rexec, and ssh connections over telnet connections, which are more difficult to configure for reliable connections.

dcom

Solaris 9 HP-UX 9 9 9 9 AIX 9 MP-RAS 9 9 9 Linux 9 9 9 9 Windows NT/2000 2

2. rsh connections are supported from Windows clients.

3

3. ssh connections are supported from Windows clients.

4

4. telnet connections are supported from Windows clients.

z/OS 9 9 9 5

5. Configure the telnet port using the AB_TELNET_PORT configuration variable in order to connect to the USS telnet server.

IRIX 9 9 9 9 Tru64 UNIX 9 9 9 9


OT COPY

Upgrade guide18 About this release > Upgrade guide AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

INTEROPERABILITYWITH EARLIER

VERSIONS

Some Co>Operating System releases improve the internal protocols used for communicating between computers during graph execution. Such improvements, however, can cause differences that prevent machines running different releases of the Co>Operating System from communicating with each other. If communication fails, graph execution fails.

The following groups of Co>Operating System releases have the same set of protocols. Computers running releases in the same group can participate in the execution of a graph. Graph execution fails if a computer is running a release outside the group.

NOTE: For releases prior to 2.11, see Ab Initio Co>Operating System Release Notes 2.13: Supplement for

Upgrading from Release 2.11.7 and earlier releases). Releases prior to 2.11 are not in any of the groups

shown above.

To upgrade your system from an older to a newer release of the Co>Operating system, you must do the following:

z Upgrade the GDE (optional)

z Install the new release

z Set AB_COMPATIBILITY

z Test the newly-installed release

z Troubleshoot any compatibility problems

The following sections describe these steps in detail.

UPGRADE THE GDE First of all, before you install the new Co>Operating System, you should consider upgrading the Graphical Development Environment (GDE) along with the Co>Operating System.

New versions of the GDE can drive old versions of the Co>Operating System. Old versions of the GDE can drive new versions of the Co>Operating System, but may not be able to use new Co>Operating System features.

Group 18 Release 2.11

Group 19 all Releases 2.11-mn

Group 20 Release 2.12, all Releases 2.12-mn, and Release 2.13


ide 19

INSTALL THE NEWCO>OPERATING

Install the new Co>Operating System and any associated software.AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Upgrade gu

SYSTEM RELEASE Even if you have done this before, please read the Ab Initio Installation Guide for the Co>Operating System and Database Package for your platform, particularly the section on configuring your operating system. This document resides on each Co>Operating System CD and can be read by popping the CD into a Windows computer. Or you can request the document from Ab Initio Customer Support at the following address:

[email protected]

When requesting this document, please identify in full the operating system you are using.

Please note that the software is much larger than it used to be. Before installing, check the exact size as listed in the Installation Guide.

Installing the newrelease on UNIX or

z/OS

On UNIX or z/OS, we suggest that you leave your old release of the Co>Operating System installed until you have thoroughly tested your applications under the new release. It is perfectly possible (and easy) to run some applications under the old release and some under the new, even at the same time!

The installation script by default installs each release into its own separate directory. If, for example, your old release was 2.1.22 and you are upgrading to, say, 2.9.11, and if you install into the default directory /usr/local, there will be two install directories

/usr/local/abinitio-V2-1-22/usr/local/abinitio-V2-9-11

and (typically) a symbolic link

/usr/local/abinitio

pointing to one of them when you have finished. The installation script will offer to change the symbolic link to point to the release it just installed, or you can do this later by hand. Now, by setting:

z PATH to the install directorys bin directory

z AB_HOME to the install directory of the desired release

you can run an application under whichever of the two releases you please.

When doing this, if you are running the application on more than one computer, make sure that the AB_HOME values set in .abinitiorc files are also correct.


OT COPY

Installing the newrelease on

On Windows NT/2000, leaving the old release installed is slightly trickier. By default, the new release will install itself into C:\AbInitio and will uninstall any old release it finds in that location. 20 About this release > Upgrade guide AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Windows NT/2000 To leave the old release in place, you will have to specify a different directory to install into when you run Install Shield to do the new install.

Once the new release is installed, you can use the release of your choice by setting the AB_HOME and PATH environment variables to C:\AbInitio (or whatever the path of the desired installation directory is) and the path of the installations bin directory respectively, similarly to the procedure described for UNIX above.

SET AB_COMPATIBILITY

Make sure the configuration variable AB_COMPATIBILITY is set correctly for your applications.

When you deploy a graph from the GDE, the GDE includes a setting of AB_COMPATIBILITY near the top of the script. This setting is intended to ensure that the deployed script performs identically under new versions of the Co>Operating System.

You should keep in mind the following about AB_COMPATIBILITY:

z By default, AB_COMPATIBILITY is not sticky.

In other words, redeploying a graph after upgrading the Co>Operating System resets AB_COMPATIBILITY to the release number of the new Co>Operating System.

You can set AB_COMPATIBILITY by entering an explicit release number under Co>Operating System Version in the Host Profile dialog box. To open the Host Profile dialog box, select Run > Settings in the GDE, click the Host tab, and click Edit.

z Versions of the GDE prior to 1.4.14 did not set AB_COMPATIBILITY.

z Hand-written mp scripts may or may not set AB_COMPATIBILITY.

TEST Using the new Co>Operating System, run your existing graphs with test inputs and verify that their output is still correct. Note that this is the most important step of all, although a discussion of how to test applications is beyond the scope of this document.

COMPATIBILITYISSUES

The following list describes ways in which the behavior a graph exhibited under an older Co>Operating System may change after you have upgraded to the present release of the Co>Operating System.


ide 21

NOTE: If you are upgrading from a pre-2.12.2 release of the Co>Operating System to Release 2.13, you

should refer to the document Ab Initio Co>Operating System Release Notes 2.13: Supplement for AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Upgrade gu

Upgrading from Release 2.11.7 and earlier releases.

Each item in the following list is divided into several sections, as follows:

z Summary: A one-line summary of a possible change in behavior.

z Upgrading from: This section identifies the previous Co>Operating System release, compared to which (after upgrading to the current version) you may experience the behavior change described in the Symptom section.

z Configuration variables: For some items in the regressions list, you can restore the behavior of the new Co>Operating System to what it was prior to the introduction of the new feature or behavior.

Where this technique is possible, a list of the configuration variable(s) you can set to obtain the old behavior is given in this section.

In most cases the operative configuration variable is AB_COMPATIBILITY; you set its value to the number of the Co>Operating System release whose behavior you want to emulate. In some other cases there is a configuration variable named for the new feature, which you can set to false to obtain the pre-feature behavior.

See Appendix A, Notes on administration in the Co>Operating System Administrator's Guide for information about setting configuration variables. See alsoSet AB_COMPATIBILITY on page 20.

z Symptom: The change(s) in behavior that can occur when you run graphs with the new release.

z Explanation: An explanation of the possible change in behavior.

z Solution: What to do if the change in behavior is a problem.


OT COPY

List ofcompatibility

The following list describes ways in which a graph may behave differently when running under the new release of the Co>Operating System as compared to an older release. 22 About this release > Upgrade guide AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

issues Summary: Install C/C++ runtime patches; recompile custom C++ code. Upgrading from: Any.

Configuration variables: None.

Symptom: Error messages from the operating systems runtime linker, particularly those about undefined symbols, whenever you try to do almost anything with the Co>Operating System.

Explanation: Two possibilities. (1) You did not install an OS patch that is required. (2) You upgraded the Co>Operating System without recompiling some C++ custom codecode at your site that is written using Ab Initio APIs.

Solution: (1) Read the Co>Operating System installation notes section on your operating system. You can find the installation notes on the Co>Operating System CD, or you can request them from [email protected]. (2) Recompile your C++ code against the new Co>Operating System. Also make sure that AB_LIB_PATH is set correctly when you run your graphs.

Summary: Lookups fail in files partitioned with one or more packed decimal keys.

Upgrading from: 2.12 to 2.13.

Symptom: A lookup done in Release 2.13 of the Co>Operating System of a partitioned file that has one or more packed decimals in the key will fail, if the partitioned file was created with Release 2.12 of the Co>Operating System.

Explanation: Co>Operating System Release 2.12.2 contained a regression which caused packed decimal fields to be hashed as voids instead of as values.

Note that patch 070, and any later patch, to Release 2.12.2 corrects the erroneous hashing behavior. To find out whether you have this patch installed for 2.12.2, you can execute the command

m_env -features

or examine the contents of your $AB_HOME/MANIFEST file.

Solution: Repartition the file, using Release 2.13.

Configuration variables: To re-obtain the (erroneous) 2.12 behavior, set the configuration variable

AB_PACKED_DECIMAL_HASH_AS_VOID

to true. This will allow data sets partitioned on erroneous hash values to be used. The default value for this configuration variable is false, (i.e. use the correct hashing behavior).


ide 23

Summary: The m_db_gencfg -password option has no effect.

Upgrading from: 2.12 to 2.13.AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Upgrade gu

Symptom: The m_db_gencfg utility does not generate the

password: password_string

field in a database configuration file, even though you specified the -password option.

Explanation: Beginning in Release 2.12, the Co>Operating System supports the use of encrypted passwords for remote connections. This includes connections to databases. Thus, specifying -password password to m_db gencfg produces no result for this field in the generated .dbc file.

Solution: You should generate an encrypted password using the m_password utility, then specify this encrypted form, using the -encrypted_password flag, to m_db gencfg.

For further information, see Password encryption in Chapter 2 of Ab Initio Co>Operating System Release Notes 2.12.2 (AB 439).


Summary: Some GDE scripts generated before Release 2.13 will not run under Release 2.13.

Upgrading from: Any.

Symptom: Some old GDE scripts will not run under Release 2.13 of the Co>Operating Systeminstead, they exit with an error during initialization.

Explanation: Beginning in Release 2.13, scripts are more carefully checked for references to unreadable or missing files. Prior to Release 2.13, scripts generated bythe GDE sometimes had a setting of the AB_CONFIGURATION configuration variable to a non-existent file. This reference to the non-existent file is detected under Release 2.13 and a script with such a reference in it will not run.

Solution: You can either regenerate the script, or edit the script and remove the AB_CONFIGURATION setting, or set the AB_COMPATIBILITY configuration variable to a release number prior to 2.13.

Configuration variables: AB_CONFIGURATION.


OT COPY

Summary: Assignment of different-size vectors is not allowed.

Upgrading from: Any.24 About this release > Upgrade guide AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Symptom: Assignment of a vector of one size to a vector of a different size results in a runtime error.

Explanation: In some releases prior to 2.13, assignment of different-size vectors was (incorrectly) sometimes permitted. This dangerous practice is no longer possible in Release 2.13.

Solution: Whenever you assign vectors, the number of elements in the source vector must equal the number of elements in the target vector.

For cases where you cannot avoid having vectors of different sizes, but you must make assignments between them, see Assignments between vectors of different size on page 26 below for a way to do this.


Summary: A final_log_output() function in the Read Multiple Files and Write Multiple Files components produces no output in the log file.

Upgrading from: Prior releases to 2.13.

Symptom: When defined and called in a Read Multple Files or Write Multiple Files component, the final_log_output() function produces no output in the log file.

Explanation: A regression in releases prior to Release 2.13 resulted in the final_log_output() function not working in these components.

Solution: Upgrade to the latest version of 2.13.



ide 25

Summary: Using numeric byte specifiers with Unicode code point specifiers, or in U"" AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Upgrade gu

constants, is now an error. Similarly, using Unicode code point specifiers in A"" or E"" constants is now an error.

Upgrading from: Any.

Symptom: Error messages such as the following examples:Error in expression U"ab\x0a":"\x" is not allowed in U string constants.

Error in expression "ab\x0a\u20ac":Mixing "\x" and "\u" is not allowed in string constants.

Error in expression A"\u0020":"\u" is not allowed in A string constants.

Explanation: Beginning in Release 2.13, numeric byte specifiers (\xhh or \nnn) cannot be used in string constants in which Unicode code point specifiers (\uhhhh) are also being used. Similarly, numeric byte specifiers cannot be used in U"" constants, and Unicode code point specifiers cannot be used in A"" or E"" constants.

Solution: Rewrite the string constant to use appropriate specifiers.


Summary: m_queue create now issues a warning message if no subscriber is specified.

Upgrading from: Release 2.12 and prior releases

Configuration variables: AB_WARN_NO_SUBSCRIBER, AB_COMPATIBILITY

Symptom: Calls to m_queue create that do not specify a subscriber now issue warning messages like the one shown in the example below:

$ m_queue create foom_queue create fooWARNING: you should specify at least one Subscriber along with creating the queue: either as an optional argument to "m_queue create"; or by following "m_queue create" with "m_queue \

subscribe".

Solution: Specify at least one subscriber (id in the synopsis below) to the call:

m_queue create [-f] queue_path [id ...]

You can obtain the previous behavior by setting the AB_WARN_NO_SUBSCRIBER configuration variable to false (the default is true). Or you can set AB_COMPATIBILITY to a release number earlier than 2.13.


OT COPY

Summary: Accuracy of conversions from real to decimal types has been improved26 About this release > Upgrade guide AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

ASSIGNMENTSBETWEEN VECTORS

OF DIFFERENTSIZE

Here are some techniques for performing assignments between vectors that have different numbers of elements.

If vec1 has N elements and vec2 has M elements, you can assign from vec1 to vec2 with a for loop:

for (i, i < M)vec2[i] = if (i < N) vec1[i] else vec2[i];

This loop behaves safely no matter which of the vectors is the larger.

Upgrading from: Release 2.12 or earlier.

Symptom: DML conversions from real numbers to decimals may yield slightly different results in graphs running with the Release 2.13 Co>Operating System, as compared with graphs running with earlier versions of the Co>Operating System.

Explanation: A slight inaccuracy in the real-to-decimal DML conversion method was corrected in Release 2.13. As a result, you may see slightly different results in graphs running with Release 2.13 where such conversions occur, as compared with results obtained running with earlier versions of the Co>Operating System.

Solution: If you want to obtain the previous (inaccurate) behavior in real-to-decimal conversions, set the AB_ACCURATE_DOUBLE_CONVERSIONS configuration variable to false. (By default, the variable is set to true.)

Configuration variables: AB_ACCURATE_DOUBLE_CONVERSIONS

Summary: Validation is now done on values assigned to dates.

Upgrading from: Release 2.12 or earlier.

Symptom: Operations involving certain values assigned to date types which succeeded in previous releases now produce an error.

Explanation: Validation checking is now done on values assigned to dates (whether directly assigned to a date type, or assigned using the encode_date( ) function). Invalid values, such as a month above 12 or a year greater than 9999, will produce an error.

Solution: Make sure you are assigning valid values to dates.



ide 27

If vec2 is larger than vec1, the loop will assign all N elements of vec1 to the first N elements of vec2, and then will go on to re-assign their original values to the remaining elements of vec2. If AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY About this release > Upgrade gu

vec2 is smaller than vec1, the loop will only assign the first M elements of vec1 to vec2.

Alternatively, you can express this as a loop expression:

vec2 = for (i, i < M) :if (i < N) vec1[i] else vec2[i];

If, rather than retaining the current value of the destination, you want to use some padding value, you can do that too:

for (i, i < M)vec2[i] = if (i < N) vec1[i]else other-expression;

or:

vec2 = for (i, i < M) :if (i < N) vec1[i] else other-expression;

This approach is necessary in transform rules. For example:

out.vec ::for (i, i < output_len) :

if (i < input_len) in.vec[i]else some-expression;


13 29AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.

2>

New features in Release 2.13This chapter describes the new features introduced in Release 2.13 of the Co>Operating System.

The chapter is organized in the following sections:

z New DML functions (page 30)

z Other new DML features and changes (page 39)

z Co>Operating System changes and new features (page 46)

z New component features (page 54)

z Platform-specific features (page 70)

z Database package new features (page 71)

z Other database package new features and information (page 79)


OT COPY

New DML functions30 New features in Release 2.13 > New DML functions AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

NEW FUNCTIONS The following DML functions are new in this release of the Co>Operating System:z hash_MD5( )

Creates the MD5 hash value of a DML object.

z hash_SHA1( )

Creates the SHA-1 hash value of a DML object.

NEW AB_CHECK_BINARY_INTEGER_CONVERSION CONFIGURATION VARIABLE

The AB_CHECK_BINARY_INTEGER_CONVERSION configuration variable is introduced in this release.

z If this variable is set to true, the Co>Operating System will check that values being stored into DML binary integers are not going to be truncated. If the variable is set to true and you attempt to store an out-of-range value into a DML binary integer (which would result in truncation), an error will occur.

z The default value is false.

NEW reinterpret( )FUNCTION

The new function reinterpret( ) reinterprets the underlying raw data as a DML object of the calls destination type. The function works like reinterpret_as( ), but it does not take the target type as its argument. Instead, it uses the expected return type as the target type.

The call

(t) reinterpret(x)

is equivalent to

reinterpret_as(t, x)

, and the call

(t) reinterpret(x, off)

is equivalent to:

reinterpret_as(t, x, off)


ns 31

NEW BUILT-IN FUNCTIONS string_to_hex AND string_from_hexAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > New DML functio

The built-in functions string_to_hex and string_from_hex are introduced in this release of the Co>Operating System.

Syntax forstring_to_hex

string string_to_hex ( in_field )

The function returns a string representation of the hexadecimal number representation of the in_field. The value of each source byte is represented by a pair of digits in the returned string.

Argument The in_field can be:z A fixed-length, or variable-length delimited, or length-prefixed string

z A fixed-length or length-prefixed void

Examples m_eval "string_to_hex('abc')" "616263"

m_eval "string_to_hex('0123')" "30313233"

Note the two-character string hexadecimal representation (30) of the first character (0) in the in_string.

Syntax forstring_from_hex

string string_from_hex ( in_field [, allow_odd_sizes] )

For each two hexadecimal characters in in_field, the function returns a single output byte with the equivalent value. The size of the output string is one-half that of the in_field.

A special case occurs if the in_field contains an uneven number of characters. An uneven in_field is rejected by default, unless the optional allow_odd_sizes argument is specified with the value 1.

Arguments z in_fieldThe in_field can be either of the following:

z A fixed-length, or variable-length delimited, or length-prefixed string

z A fixed-length or length-prefixed void

z allow_odd_sizes


OT COPY

If in_field contains an uneven number of characters, and allow_odd_sizes is specified with the value 1, the function will convert the in_field (instead of rejecting it, which is what happens 32 New features in Release 2.13 > New DML functions AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

by default). The final odd character in in_field is paired with an implied trailing zero.

Examples m_eval "string_from_hex('616263')" "abc"

m_eval "string_from_hex('6162637', 1)" "abcp"

NEW BUILT-IN FUNCTIONS scanf_double AND scanf_long

The built-in functions scanf_double and scanf_long are introduced in this release to complement the built-ins scanf_float and scanf_int.

Syntax forscanf_double

double scanf_double ( string format_string, string str )

Arguments format_stringA format for a double type in C scanf syntax

strA string

Syntax forscanf_long

long scanf_long ( string format_string, string str )

Arguments format_stringA format for a long type in C scanf syntax

strA string

Examples As with other scanf functions, some knowledge of the C scanf routine is required; there may be platform-specific ways of identifying double and/or long values for scanf.

An example of using scanf_double:

scanf_double("%lf", "3.141592653589793") 3.141592653589793

An example of using scanf_long:

scanf_long("%lld", "123456789101112") 123456789101112


ns 33

NEW BUILT-IN FUNCTION string_representationAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > New DML functio

A new built-in function string_representation has been added. This function returns the text form of the object supplied (the same text form as is produced by m_dump or by the logging mechanism).

Syntax string string_representation ( object )

Arguments objectA single object of any type, or combination of objects

of different types

Examples string_representation(123) "123"

A more complicated example:

string_concat("The string_representation is: ", string_representation([record x 23 s "hi there" y 92]))

"The string_representation is: [record\n x 23\n s \"hi there\"\n y 92]"

NEW DATEFUNCTIONS

This release of the Co>Operating System contains the following new functions:

z date_add_months

z date_difference_days

z date_difference_months

z datetime_add_months

z datetime_difference_days

z datetime_difference_hours

z datetime_difference_minutes

z datetime_difference_months

z datetime_difference_seconds

The functions are described separately below.

date_add_months

Returns the internal representation of a date resulting from adding (or subtracting) a number of months to the specified date.


OT COPY

Syntax int date_add_months ( date date_in, int months)34 New features in Release 2.13 > New DML functions AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Arguments date_in A date

months The number of months to add to date_in. To subtract

months, specify a negative number

Details If date_in is the last day of a month, the returned value will also represent the last day of the calculated month. For example:

z If date_in is February 28, 2006 and the function adds five months, the returned value will represent July 31, 2006.

z If the date_in is August 31, 2005 and the function subtracts two months (months is -2), the returned value will represent June 30, 2005.

NOTE: If date_in is a datetime type, the function converts it to a date.

date_difference_days

Returns the number of days between two dates.

Syntax double date_difference_days ( date date1, date date2 )

Arguments date1 A date value

date2 A date value

Details The date_difference_days function returns a double value representing the number of days between date1 and date2. If date1 is earlier than date2, the returned value is negative. If date1 or date2 is a datetime type, it is converted to a date type.

date_difference_months

Returns the number of months between two dates.

Syntax double date_difference_months (date date1, date date2)


ns 35

Arguments date1 A date valueAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > New DML functio

date2 A date value

Details The date_difference_months function returns a double value representing the number of months between date1 and date2. If date1 is earlier than date2, the returned value is negative.

If the day values in date1 and date2 are the same, or if they are both the last day of the respective month, the value returned is a whole number; otherwise the returned value is a fractional number.

If date1 or date2 is a datetime type, it is converted to a date type.

datetime_add_months

Returns the result of adding (or subtracting) a given number of months to the value of a datetime. The returned value is a datetime in the form: YYYYMMDDHH24MISSnnnnnn.

Syntax int datetime_add_months ( datetime date_in, int months)

Arguments date_in A datetime

months The number of months to add to date_in. To subtract

months, specify a negative number

Details If date_in is the last day of a month, the returned value will also represent the last day of the calculated month. For example:

z If date_in is February 28, 2006 and the function adds five months, the returned value will represent July 31, 2006.

z If the date_in is August 31, 2005 and the function subtracts two months (date_in is -2), the returned value will represent June 30, 2005.

NOTE: If date_in is a date type, the function converts it to a datetime.

datetime_difference_days

Returns the number of days between two datetime types.


OT COPY

Syntax double datetime_difference_days ( datetime date1, datetime date2 )36 New features in Release 2.13 > New DML functions AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

Arguments date1 A datetime value

date2 A datetime value

Details The datetime_difference_days function returns a double value representing the number of days between date1 and date2. If date1 is earlier than date2, the returned value is negative. If date1 or date2 is a date type, it is converted to a datetime type.

datetime_difference_hours

Returns the number of hours between two datetimes.

Syntax double datetime_difference_hours ( datetime date1, datetime date2 )



Details The datetime_difference_hours function returns a double value representing the number of hours between date1 and date2. If date1 is earlier than date2, the returned value is negative. If date1 or date2 is a date type, it is converted to a datetime type.

datetime_difference_minutes

Returns the number of minutes between two datetimes.

Syntax double datetime_difference_minutes (datetime date1, datetime date2)



Details The datetime_difference_minutes function returns a double value representing the number of minutes between date1 and date2. If date1 is earlier than date2, the returned value is negative. If date1 or date2 is a date type, it is converted to a datetime type.


ns 37

datetime_difference_monthsAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > New DML functio

Returns the number of months between two datetimes.

Syntax double datetime_difference_months (datetime date1, datetime date2)



Details The datetime_difference_months function returns a double value representing the number of months between date1 and date2. If date1 is earlier than date2, the returned value is negative.

If the day values in date1 and date2 are the same, or if they are both the last day of the respective month, the value returned is a whole number; otherwise the returned value is a fractional number.

If date1 or date2 is a date type, it is converted to a datetime type.

datetime_difference_seconds

Returns the number of seconds between two datetimes.

Syntax double datetime_difference_seconds (datetime date1, datetime date2)



Details The datetime_difference_seconds function returns a double value representing the number of seconds between date1 and date2. If date1 is earlier than date2, the returned value is negative.

If date1 or date2 is a date type, it is converted to a datetime type.

NEW DML evalFUNCTION

This release includes a new DML function: eval. Its syntax is as follows:

Syntax object eval ( record context, string expression )


OT COPY

Arguments contextRecord context in which to evaluate the expression38 New features in Release 2.13 > New DML functions AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

expressionAn expression to evaluate

The eval function takes the DML string expression passed to it, evaluates it in the given record context, and returns the result.

The record context is used to resolve variable references within the expression: variable references in the string expression are resolved to fields of the same name in the record context.

As with the built-in function allocate and with recursive function calls, an explicit cast may be required to specify the type of the result of a call to this function. Examples of this are given below.

Examples In the following simple example, the value of the field a is added to itself and returned as an int:(int)eval([record a 2], "a+a")

4In the following example, the contents of the two records s and t are concatenated and returned as a string:

(string(""))eval([record s "xyz" t "lmno"],

"string_concat(s, s, t)")

"xyzxyzlmno"

A more interestingexample

The context object can be constructed using any available variables, including transform function inputs, local variables, global variables, or arbitrary expressions. For example:

(string(""))eval([record in in0

l loc1.q

g glob2],

lookup("Scoring Table",

in.classification).expr)

Here, eval evaluates an expression found by looking up in.classification in the lookup file Scoring Table. Expressions in that file may reference variables in, l, and g, which are given the values of in0, loc1.q, and glob2, respectively, in the record context.

NOTE: The eval function can be expensive in terms of performance, especially when a large number of

different expressions are passed into it.


es 39

Other new DML features and changesAB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > Other new DML features and chang

VALIDATION NOWDONE ON DATEASSIGNMENTS

Beginning in this release of the Co>Operating System, validation checking is performed on all values assigned to dates (whether assigned directly to date types, or assigned using the encode_date( ) function). Invalid values, such as a month greater than 12 or a year greater than 9999, will produce an error.

UNSIGNED ZONEDDECIMALS

Support for unsigned zoned decimals has been added in this release of the Co>Operating System.

An unsigned zoned decimal is a decimal number in which the sign nibble has the value 0xf0. The size of the number includes the sign nibble. For example, the decimal value 12.3 could be represented in a field declared as

ebcdic decimal(4,1,zoned,unsigned)

with the following hexadecimal values:

0xf1 0xf2 0xf3

NEW VSPECIFIER IN

DELIMITED DATESAND DATETIMES

The V specifier for delimited dates and datetimes is new in this release of the Co>Operating System.

Specifying V in a delimited date or datetime format means that the data is allowed to consist of from 1 to the maximum format size number of spaces.

For example, with a date type specification as follows:

date(VYY/MM/DD)(,)

the following data would be valid:

77/1/1

Note that if you use this specifier, it must appear as the first specifier in the format string.

LENGTH-PREFIXEDVECTORS

Length-prefixed vectors are available in this release of the Co>Operating System.

A length-prefixed vector (sometimes called a varvector) has a variable number of elements. Length-prefixed vectors can occur in a record or in a variable.

Length-PrefixedVector Syntax

type_specifier [ type_specifier ]


OT COPY

type_specifier A type specifier describing a vector element. 40 New features in Release 2.13 > Other new DML features and changes AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

About the LengthPrefix

In a length-prefixed vector, the length of the vector value is stored in a prefix; the prefix is made up of the bytes preceding the vector value. The total size of a length-prefixed vector is the number of bytes that store the prefix, plus:

z (for a variable-length character set) The number of bytes indicated by the value stored in the prefix

z (for a fixed-length character set) The number of characters indicated by the value stored in the prefix

NOTE: Because the data in the prefix must be interpreted to determine the vector length, an error in

specifying the prefix can make it difficult or impossible to determine the size of a field or record. Such an

error does not cause records to be written to a graph components log or reject port, but instead causes

the component to shut down.

Varvector The words varvector, varstring, varchar, and varvoid are not DML keywords, although the terms varstring and varvoid do appear in the Data Type drop-down menu in the GDEs Record Format Editor to refer to a length-prefixed string and length-prefixed void, respectively.

Length-PrefixedVector Example

Given the following raw data:

And the following vector type specifier:

recordstring(",") [decimal(2)];

end

The resulting data record is:

[vector "ab", "def"]

type_specifier A type specifier that describes the prefix that stores the length (number of elements) in the vector. The type specifier must be a fixed-size DML numeric type .

0 2 a b , d e f ,


es 41

KEY-BASEDOPERATIONS

Beginning in this release, key-based operations are extended to a wider variety of types. In previous releases, neither vector types nor binary integer types of other than 1, 2, 4, or 8 bytes AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > Other new DML features and chang

EXTENDED TOMORE TYPES

could be used in keys. Those restrictions are now lifted.

The rules for comparisons and other key-based operations on integer(3), integer(5), integer(6), and integer(7) are as for other integer types.

Where keys are used to order data (as in Sort or Merge), vectors are compared as follows.

z Vectors are compared element-by-element according to the comparison rules for the vectors element type.

z Earlier elements are more significant than later elements.

z If two vectors of different lengths have equal elements up to the length of the shorter vector, the shorter vector is considered less than the longer vector.

z A zero-length vector is considered less than any non-zero length vector.

z If an entire vector is NULL, it is considered less than any non-NULL vector.

ASSIGNMENT TOUNION MEMBERS

Beginning in this release, you can specify union members as destinations in transform rules.

The changes are as follows:

z Union members may now appear on the left hand side of :: rules.

z The rules for a unions members are attempted until one of them succeeds (that is, generates a non-NULL value). The order in which they are attempted is the order in which the corresponding members appear in the output type, just as with fields of records. (As a practical matter, this order will usually be irrelevant, since conditional expressions will usually be used to choose assigning one member over another.)

z At least one member of a union must be assigned.

z For a union member with complex substructure (for example, a nested union or record), there may be rules that separately assign pieces. All of those rules must succeed for the member to be assigned from the corresponding set of rules.


OT COPY

Example 1 Given the following output type:record42 New features in Release 2.13 > Other new DML features and changes AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

string(1) k;union

string(2) a;decimal(2) b;void(2) c;

end u1;end

consider this transform function:

out :: reformat(in) =begin

out.k :: in.k;out.u1.a : : if (f(in.k)) in.a;out.u1.b : : if (g(in.h)) in.b;out.u1.c :1: if (h(in.k)) in.q;out.u1.c :2: in.r;

end;

Depending on the value of in.k, the union will be populated either with a string(2) value from in.a, a decimal(2) value from in.b, or a void(2) value from either in.q or in.r. If more than one of the conditions f(in.k), g(in.k), or h(in.k) is true, then the union will be populated with the value assigned to the first-declared member.

Example 2 Given the following output type:record

unionstring(2) a;decimal(2) b;record

string(1) r1, r2;end r;void(2) c;

end u2;string(1) newline = '\n';

end


es 43

consider this transform function:

out :: reformat(in) =AB INITIO CONFIDENTIAL AND PROPRIETARY DO NOT COPY New features in Release 2.13 > Other new DML features and chang

beginout.u2.a :: if (in.k == 's') in.a;out.u2.b :: if (in.k == 'd') in.b;out.u2.r.r1 :1: if (in.k == 'z') 'Z';out.u2.r.r1 :2: 'X';out.u2.r.r2 :1: if (in.k == 'x') 'Y';out.u2.c :: '!!';

end;

When in.k is s, the union will be populated as a string(2) from in.a.

When in.k is d, the union will be populated as a decimal(2) from in.b.

However, when in.k is z, the union will be populated as a void(2) from the string constant '!!', from the out.u2.c rule. Note that while the field out.u2.r.r1 could be populated in that case, its neighbor out.u2.r.r2 cannot, and so processing will fall through to the rules for the next union member, out.u2.c.

NULLS INCONTROL FLOW

STATEMENTS

In this release, the behavior of NULLs which occur in control flow statements of user-defined functions has been changed.

In previous releases, if a NULL occurred in the evaluation of a condition in an if, for, while or switch statement within a user-defined function, the function immediately returned NULL. Statements after the NULL evaluation occurred were not processed. Beginning with the present release, a NULL in such a control flow causes an abort.

The old behavior can be retained if the environment variable

AB_NULLS_FROM_CONTROL_FLOW_ARE_ERRORS

is set to false.

CHANGES INab_type_info

UTILITY

The following features have been added to the ab_type_info utility in this release of the Co>Operating System:

z The utility now takes input from standard input if

-file -

is specified for the input type.

z The utility has two new command line options:

z -field-delimiter chr


OT COPY

Specifies the separator used between attributes. If not specified with this oprion, the separator defaults to tab.44 New features in Release 2.13 > Other new DML features and changes AB INITIO CONFIDENTIAL AND PROPRIETARY DO N

z -last-field-delimiter chr

Specifies the separator used between fields. If not specified with this option, the separator defaults to newline.

If NUL is specified for chr in either case, the NUL character is used.

z When the -flatten option is specified and a record type has conditional fields, the conditions are pushed down to the fields of subrecords.

ESCAPES IN U"..."AND OTHER

STRINGCONSTANTS

The hexadecimal byte character escape notation (\x) and octal byte character escape notation (\0 - \7) must not be used within U"..." string constants.

Similarly, the Unicode code point character escape notation (\u) must not be used within A"..." or E"..." string constants.

Either notation can, however, be used in string constants that do not use the A, E, or U syntax.

Note that you cannot use byte escapes and Unicode escapes together in the same string constant, no matter what kind it is.

Note also that using a \u escape within a "..." string constant, where the code point is greater than \u00ff, will result in the string constants becoming a Unicode string, just as if it had originally been written with the U"..." syntax.

COBOL-TO-DML cobol-to-dml (and the Import Cobol... functionality in the GDEs Record Format Editor) now understands values of the form

ALL string-literal

where the string literal is one character long.

NAMEDCONSTANTS IN

DML

Named constants can be defined at the top level of a package. The syntax for declaring a constant is much like that for defining a global variable, with the reserved word constant used in place of let. Here are some constant declarations:

constant double pi = 3.14159265358979323846e0;constant double r = 10;constant double r_squared = math_pow(r, 2e0);constant string("") r_units = "miles";constant double fraction = 0.5;constant double area =


es 45

fraction * pi * r_squared;constant string("") message = string_concat("The

cooperating system release notes

Documents