What is ODI

Oracle Data Integrator

What is used for

Oracle Data Integrator (ODI) features an active integration platform that includes all styles of data integration: data-based, event-based and service-based. ODI unifies silos of integration by transforming large volumes of data efficiently, processing events in real time through its advanced Changed Data Capture (CDC) framework, and providing data services to the Oracle SOA Suite. It also provides robust data integritycontrol features, assuring the consistency and correctness of data. With powerful core differentiators - heterogeneous E-LT, Declarative Design and Knowledge Modules - Oracle Data Integrator meets the performance, flexibility, productivity, modularity and hot-pluggability requirements of an integration platform.

What is E-LT?

Traditional ETL tools operate by first Extracting the data from various sources, Transforming the data in a proprietary, middle-tier ETL engine that is used as the staging area, and then Loading the transformed data into the target data warehouse, integration server, or Hadoop cluster. Hence the term ETL represents both the names and the order of the operations performed, as shown in Figure

E-LT moves the data transformation step to the target RDBMS, changing the order of operations to: Extract the data from the source tables, Load the tables into the destination server, and then Transform the data on the target RDBMS using native SQL operators. Note, with E-LT there is no need for a middle-tier engine or server as shown

Understanding the Oracle Data Integrator Component Architecture

The Oracle Data Integrator platform integrates in the broader Fusion Middleware platform and becomes a key component of this stack. Oracle Data Integrator provides its run-time components as Java EE applications, enhanced to fully leverage the capabilities of the Oracle WebLogic Application Server. Oracle Data Integrator components include exclusive features for Enterprise-Scale Deployments, high availability, scalability, and hardened security. Figure shows the ODI component architecture. Figure Oracle Data Integrator Component Architecture

The Oracle Data Integrator Repository is composed of a master repository and several Work Repositories. Objects developed or configured through the users are stored in one of these repository types

There is usually only one master repository that stores the following information:

■ Security information including users, profiles and rights for the ODI platform.

■ Topology information including technologies, server definitions, schemas, contexts, languages, etc.

■ Versioned and archived objects.

The Work Repository is the one that contains actual developed objects. Several work repositories may coexist in the same ODI installation (for example, to have separate environments or to match a particular versioning life cycle). A Work Repository stores information for: ■ Models, including schema definition, datastores structures and metadata, fields and attributes definitions, data quality constraints, cross references, data lineage etc. ■ Projects, including business rules, packages, procedures, folders, Knowledge Modules, variables etc. ■ Scenario execution, including scenarios, scheduling information and logs. When the Work Repository contains only the execution information (typically for production purposes), it is then called an Execution Repository.

ODI INSTALLATION

1. Log in to the target system.2.

2. The installer requires that a certified JDK already exists on your system. For moreinformation, see the appropriate certification document for 12c (12.1.3) on the

Oracle Fusion Middleware Supported System Configurations page.3. Go to the directory where you downloaded the installation program.4. Launch the installation program by invoking java -jar from the JDK directoryIn my system, jdk is installed on below locationC:\Program Files\Java\jdk1.7.0_75\bin\java -jar odi_121300.jar.

2.Configuring a Certified Database

We already installed 12c database for this purpose.

3.Starting the Repository Creation Utility

On Windows operating systems, right-click on the desktop, then select MyComputer, then select Properties. Click Advanced System Settings in the leftcolumn, then click Environment Variables in the System Properties window.In the Environment Variables window, set JAVA_HOME to your JDK location.

Selecting a Repository Creation MethodIf you have the necessary permission and privileges to perform DBA activities on yourdatabase, select System Load and Product Load Concurrently. This procedureassumes that you have the necessary privileges.If you do not have the necessary permission or privileges to perform DBA activities inthe database, you must select Prepare Scripts for System Load on this screen. Thisoption will generate a SQL script, which can be provided to your databaseadministrator. See "Understanding System Load and Product Load" in Oracle FusionMiddleware Creating Schemas with the Repository Creation Utility

Providing Database Connection Details

Selecting ComponentsSelect Create a new prefix, specify a custom prefix, then select the Oracle DataIntegrator schema.The custom prefix is used to logically group these schemas together for use in thisdomain only; you must create a unique set of schemas for each domain as schemasharing across domains is not supported.If you are running RCU from a Standalone installation, you will see the following:

Select Oracle Data Integrator and click Next to proceed, then click OK on the dialogwindow confirming that prerequisite checking for schema creation was successful.

Specifying Schema PasswordsSpecify how you want to set the schema passwords on your database, then specify andconfirm your passwords.Task 6 Setting Custom VariablesSpecify the custom variables for the Master and Work Repository.Task 7 Completing Schema CreationNavigate through the remainder of the RCU screens to complete schema creation.When you reach the Completion Summary screen, click Close to dismiss RCU.Note: If you have already created the ODI repository using ODIStudio, uncheck the ODI entry here and create only the dependentschema. In the Configuration Wizard, after retrieving the dependentschema connectivity from STB schema, manually fill in the ODIrepository details.Tip: Make a note of the custom prefix you choose to enter here; youwill need this later on during the domain creation process.

Tip: You must make a note of the passwords you set on this screen;you will need them later on during the domain creation process.

Password:Bend1234

Configuring Oracle Data Integrator Studio

ODI StudioOracle Data Integrator Studio is a developer’s interface for configuring and managingODI. Studio is installed as part of standalone or enterprise option under the specifiedORACLE_HOME. ODI Studio provides four graphical Navigators for managing ODIartifacts: Designer, Operator, Topology, and Security.ODI Studio can also directly execute jobs on demand, and can be used fordevelopment and initial testing.

Creating the Database SchemaBefore you can configure your ODI topology, create the master and work repository

database schemas using the Repository Creation Utility (RCU)

Starting ODI StudioTo start the ODI Studio:1. Go to the ORACLE_HOME/odi/studio directory.2. Start ODI Studio.

Getting Started

From the ODI Studio menu, select File, then select New.

On the New gallery screen, select Create a New ODI Repository Login, then click OK.

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=PDBORCL)))

Creating an Agent in the Master Repository Using ODI Studio

A physical agent corresponds to a single standalone agent or a Java EE agent. Aphysical agent should have a unique name in the topology.As part of its startup sequence, an agent connects to the Master Repository to see ifthere is a physical agent defined with its name. If it finds its entry, the agent continueswith startup and, once started, it reads all the scheduled jobs of itself from therepository and starts processing.If there is no physical agent entry for an agent, then the agent startup fails.Follow the instructions in this section to create an ODI agent after you have connectedto the repository in ODI Studio.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit ProductionWith the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> show pdbs

CON_ID CON_NAME OPEN MODE RESTRICTED---------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 PDBORCL MOUNTEDSQL>

Task 1 Adding an Agent to the Oracle Data Integrator TopologyIn the navigation pane on the left side, select the Topology tab, then right-click Agents,then select New Agent.

Task 2 Specifying the Agent DetailsSpecify the new agent details as summarized in the following table:For more information about the fields in this screen, see "Creating a Physical Agent" inAdministering Oracle Data Integrator.Note: If you do not wish to configure any of the agent types, and youjust want to work with ODI Studio, then the following procedures forcreating Java EE and standalone agents are not necessary.

STILL SOME JOB IS PENDING IN THIS SECTION

Configuring the Domain for the Java EEAgentdescribes how to create and configure a Oracle Data Integrator (ODI)domain for the Java EE agent standard installation topology.

Starting the Configuration WizardTo begin domain configuration, navigate to the ORACLE_HOME/oracle_common/common/bin directory and start the WebLogic Server Configuration Wizard.On UNIX operating systems:

./config.sh

On Microsoft Windows operating systems:

config.cmd

Task 1 Selecting the Domain Type and Domain Home LocationOn the Configuration Type screen, select Create a New Domain.In the Domain Location field, specify your Domain home directory.It is recommended that you locate your Domain home in accordance with thedirectory structure summarized in "Understanding the Recommended DirectoryStructure" in Oracle Fusion Middleware Planning an Installation of Oracle FusionMiddleware, where the Domain home is located outside the Oracle home directory. Thisdirectory structure will help you avoid issues when you need to upgrade or reinstallyour software

start your per-domain Node Manager, go to the DOMAIN_HOME/bin directory.On UNIX operating systems, start the Node Manager as shown below, using nohupand nm.out as an example output file:nohup ./startNodeManager.sh > nm.out&On Windows operating systems, run:startNodeManager.cmd