kscope15 - automatic promotion and versioning with oracle data integrator

T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)

E : info@rittmanmead.comW : www.rittmanmead.com

Rittman Mead ConsultingManage your ODI Development Lifecycle Oracle Data Integrator 12c

Jérôme FrançoisseKscope15 – June 2015 – Hollywood FL

Jérôme Françoisse•Consultant for Rittman Mead

‣Oracle BI/DW Architect/Analyst/Developer•ODI Trainer•Providing ODI support on OTN Forums•ODI 12c Beta Program Member•Blogger at http://www.rittmanmead.com/blog/•Email : jerome.francoisse@rittmanmead.com•Twitter : @JeromeFr

Rittman Mead•Oracle Gold partner with offices in Europe, US (Atlanta), Australia, India and South

Africa•World leading specialist partner for technical excellence, solutions delivery and

innovation in Oracle BI•Provide consulting, training, global managed services for customers around the world•120+ consultants including 1 Oracle ACE Director and 3 Oracle ACEs•All expert in Oracle BI, DW, EPM and Analytics tech•Skills in broad range of supporting Oracle tools: OBIEE, OBIA, ODIEE, Essbase,

Oracle OLAP, GoldenGate, Exadata, Endeca, Big Data, …•Blog : http://www.rittmanmead.com/blog/•Twitter : @rittmanmead

Visual Redesign Business User TrainingOngoing SupportEngagement Toolkit

Average user adoption for BIplatforms is below 25%Rittman Mead’s User Engagement Service can help

ODI 12c Key Components•ODI Studio (the main development IDE)‣Designer Navigator‣Operator Navigator‣Topology Navigator‣Security Navigator

•ODI Agents (executing mappings and packages)•ODI Console (web-based interface)•ODI Repositories (contain definitions of models, interfaces and other objects)

DesignerModel data sources

Define projectsCreate interfaces

Release scenarios

OperatorView running interfaces and

packagesOperate production

RepositoryStandaloneAgent

Orchestrateinterfaces and scenarios

Information Systems(Databases, files, applications)

ODI Architecture

ODI 12c Key Components

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

Work Repository(Development)

Execution

Execution Repository(Production)

•ODI Studio (the main development IDE)‣Designer Navigator‣Operator Navigator‣Topology Navigator‣Security Navigator

•ODI Agents (executing mappings and packages)•ODI Console (web-based interface)•ODI Repositories (contain definitions of models, interfaces and other objects)

Physical and Logical Topology

DEV_SCOTT

HUM_RES

DEV_OP

TEST_OP

PROD_OP

Physical

Contexts Logical

How many repositories do I need?

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

Execution Repository(Test)

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

Execution

Execution Repository(Pre-Production)

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

Work Repository(Hotfix)

Execution

SecurityTopology

Versioning

Master Repository

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

SecurityTopology

Versioning

Master Repository

Execution

Execution Repository(Smoke Test)

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Export

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

- Manually

Versioning - Create a version

Versioning - Comparison

Versioning - Restore a version

Versioning – Version Browser

Automation

XKCD.com

Automation

XKCD.com

Export - Automated within a package

Export

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

ModelsProjects

Execution

SecurityTopology

Versioning

Master Repository

SecurityTopology

Versioning

Master Repository

Execution

Execution Repository(Smoke Test)

Automation

XKCD.com

ODI SDK - Scripting/*(...) Connection to the source repository(...) Creation of a collection of mappings(...) Instanciation of an OdiScenarioGeneratorImpl object and Mapping finder Object*/

// Generating Scenarios mappings = mappingFinder.findByProject(projectCode, folderName);For(Object mapping : mappings) { Mapping odiMap = (Mapping) mapping; String scenName = odiMap.getName(); OdiScenario newScen = gene.generateScenario(odiMap, scenName, newVersion); odiMap.exportToXml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding); }

ODI SDK - Scripting/*(...) Creation of a collection of the generated scenarios*/

// Exporting Scenariosfor (Object scen : scenarioCollection) { OdiScenario odiscen =(OdiScenario)scen ; export.exportToXml(odiscen, ExportPath, OverWrite, RecursiveExport, Encoding);}

ODI SDK - Scripting/*(...) Connection to the target repository(...) Creation of a list of XML files.*/

// Importing Scenariosfor (String filename : XMLFiles) { import.importObjectFromXml( import.IMPORT_MODE_SYNONYM_INSERT_UPDATE, filename, true);}

SubVersion •Revision control tool

•Checkout a project from the remote repository•Make some changes•(Add files)•Commit your changes with a message to the remote repository• In case of conflict : Manual resolution•Restore earlier versions

SubVersion – Folders structure•Trunk – Master copy of the code•Branches – Ongoing development or different versions•Merge branch in the trunk when development is done

Git•Created by Linus Torvalds•Local repository - Remote repository•Built-in branching mechanism•Built-in merging for the branches and the master copy•Pull request

GIT - Commit and Push

git add . -Agit commit -m "My commit Message"git push

•From OdiOSCommand•From Java as an OS Command•From Java using a library‣Jgit‣JavaGit‣GitHub API for Java (kohsuke)

Oracle can do it !•Lifecycle Management features•Announced at OOW 2014•Third-party tool (SubVersion? Git?)

•Oracle Data Integrator 12c – Wait for it•Oracle Data Integrator 10g or 11g – DIY or upgrade

Further reading•Blog post by Uli Bethke (@ubethke) :“ODI: AUTOMATING DEPLOYMENT OF SCENARIOS TO PRODUCTION IN ORACLE DATA INTEGRATOR”http://sonra.io/odi-automating-deployment-of-scenarios-in-oracle-data-integrator/

•Blog post by kdevendr (ODI Experts) :“Export and Importing Scenarios based on Project (or)Folder”http://odiexperts.com/export-and-importing-scenarios-based-on-project-orfolder/

•ODI SDK Public API : https://docs.oracle.com/middleware/1213/odi/reference-java-api/index.html

•David Allan (@i_m_dave) :“ODI - Accelerator Launchpad, getting Groovy” and much morehttps://blogs.oracle.com/dataintegration/entry/odi_accelerator_launchpad_getting_groovy

kscope15 - automatic promotion and versioning with oracle data integrator

Data & Analytics

web service versioning

osgi versioning and testing

extensibility and versioning

dependency management & versioning

database versioning

versioning in the cloud

semantic versioning lightning talk

release and versioning

versioning 101

· web viewintegrator agreement [symbol logo] agreement...

data versioning systems

360plus sauvegarde, corbeille, versioning et promotion pour...

versioning avec git

versioning, consistency, and agreement

maven versioning strategy (vr)

versioning for workflow evolution

row-level versioning

rittman mead consulting automatic promotion and versioning

versioning apis

contract versioning