data integration extravaganza
DESCRIPTION
TRANSCRIPT
Oracle EPM (Hyperion) Suite Data Integration Extravaganza!
Applications and Service Offerings
AdvancedEPM has successfully completed 100+ EPM/BI Implementations
Financial Consolidation
Budgeting & Planning
Business Intelligence & Analytics
Profitability Management Solutions
Data Warehousing
Infrastructure
Performance Tuning & Optimization
Business Process and Project Management
Organizational Change Management
AdvancedEPM Service Offerings
Full Lifecycle Product Implementations
Upgrades & Migrations
Managed Services
Training & Support
Center of Excellence Development
Project Management
Change Management
Gap Assessments / Health Checks
Road Mapping
Business Process Improvement
AdvancedEPM has a series of service offerings within our core focus of Oracle Hyperion Enterprise Performance Management and Business Intelligence.
Thought LeadershipWe are committed to bringing our clients the latest thinking and education around EPM & BI.
Webcasts Speaking Events Training
Join us and expand your skillset!
AdvancedEPM is proud to be offering 5-10 FREE webcasts each month focused on Oracle Enterprise Performance Management and MDM solutions.
Register today!www.advancedepm.com
AdvancedEPM is recognized by the Oracle Hyperion community and is frequently invited to speak at conferences on various EPM topics.
A few 2013 Speaking Events• Collaborate 2013• Kaleidoscope• CFO Magazine – Corporate
Performance Management Event
Not all training is created equally!
AdvancedEPM recognizes that every Oracle Hyperion training class has unique requirements. We evaluate your needs and deliver Hyperion training courses that are tailored to your learning requirements.
Technologies• Hyperion Application Link (HAL)• Data Integration Management (DIM) • Oracle Data Integrator (ODI)• Enterprise Performance Management Architect (EPMA)• Financial Data Quality Management (FDM)• Financial Data Quality Management Enterprise Edition (FDMEE)• ERP Integrator (ERPi)• Data Relationship Management (DRM)• Essbase Integration Services (EIS)• Essbase Studio (EAS)• Outline Load Utility• DATAEXPORT Command• MaxL• Windows Batch Scripting• Essbase SQL Loader• Essbase Analytics Link (EAL) for HFM
And More!!
Informatica, Oracle Data Integrator, EPMA, OutlineLoad Utility, Load Rules, Report Scripts, Calculation Scripts, Shell Scripts, Windows Batch, VBScript, @XREFS, @XWRITES, Database Partitioning, ERP Integrator, Web Services, etc., etc.
Be Happy – This Stuff is Fun
HAL• Background
– Hyperion tool for loading data and metadata to Hyperion Essbase, Financial Management, and Planning
– No longer supported by Oracle– Vignette Business Integration Studio (VBIS)
• Common Uses– Loading data (text, relational) to Essbase, Planning– Extracting outline information
• Moped vs. Sport Bike?
HAL
HAL• Use Case
– Company wants to move GL data from a relational data source or flat file and use it to update the outline in Planning
– Extract Data From Essbase/Planning
DIM• Data Integration Management• Based on Informatica PowerCenter• ETL tool similar to ODI
DIM• Can load data and metadata to Essbase,
Planning, HFM, Performance Scorecard, etc.• Can process large volumes of data and
metadata with complex transformations• Not Oracle’s strategic direction
DIM• Use Case(s):
– IT driven process passing large volumes of data at high speeds
– Complex mapping logic is required– Multiple workflows – Robust Process Monitoring– Web Service(s) Enablement (WSDL, SOAP, UDDI)
ODI• Oracle Data Integrator• Very flexible tool• Knowledge modules for interacting with
systems (Essbase, Planning, SQL Server, Oracle, Flat file, etc.)
ODI• Can load data and metadata to Essbase,
Planning, HFM, etc.• Can load data warehouses, ERP systems and
most databases• Strategic data integration tool from Oracle
ODI• Use Case
– Upstream data is in relational databases accessible via JDBC driver, wants to combine data from a file coming in via FTP and use that to load to a Hyperion cube, and send an email out on failure.
EPMA• Can manage metadata and applications (*)
within a graphical interface• Dimension Library centralizes hierarchies for
different dimensions• Can move data with Data Synchronization• Interface Tables • Application Library
EPMA• Manage Essbase, Planning, HFM in one place• Mostly deals with metadata• Provides validation functionality• Provides limited data mapping / integration
Define properties for all products for each member
Dimension Library
Graphical designer for Oracle EPM ETL processes
Data Synchronization
EPMA• Use Case
– Administrator already uses EPMA, deploys metadata to various applications
– Ad hoc data sync across environments– Quickly build a new application for a specific
purpose by using metadata and data from existing applications
AD-HOC CAGE MATCHEPMA vs. DRM!!
What’s the difference?Feature DRM EPMA
Property inheritance
Redeploy Oracle EPM applications
Customized property value validation
Customized structural validation
Member formula validation
Metadata versioning
Rollback to point-in-time
Share members across dimensions
Enforce custom business rules
= Limited Support
= Full/Advanced Support
= Not Supported
What’s the difference?Feature DRM EPMA
Interface with non-Oracle EPM applications
Customizable user interface
Custom property creation/definition
Workflow/approvals
Flexible output format
Transaction history
= Limited Support
= Full/Advanced Support
= Not Supported
What’s the difference?
Same Master Data Management Capabilities?
EPMAWatercolor (Kindergarten)
DRMStarry Night (Van Gogh)
Better Together – We Can All Get Along!!
DRM&
EPMA
How Can They Work Together?
Data Relationship Management
(1)Manage Enterprise
hierarchies and attributes
EPM Architect
HFM / HP / Essbase
(2)Packaged
Integration updates EPM hierarchies and
attributesin EPMA
(3)Manage EPM
applications and share dimensions across
applications
(4)Validate and deploy
dimensions to applications
FDM• Financial Data Quality Management• Load HFM, Essbase & Planning from flat files• SQL based, business driven• Highly customizable with VB Script• Self-service or automated data loading• Allows drill back from target systems
FDM
FDM
• Use Case– Enable SME’s to map their own data– Real time validation & verification – SOX Compliance & Certification– Industrial strength ETL not required– Sarbanes Oxley Certs
FDM
ERPi• Introduced in 11.1.1.3• Pulls directly from source ERPs• Load metadata to HFM, Planning and Essbase• Combined with FDM, loads data to HFM,
Planning or Essbase• Facilitates user drill back and write-back to
source ERP from HFR or Smart View
ERPi
ERPi
• Use Case– Enhance functionality of existing FDM– With supported source ERPs, create robust 2-way
data flow for reporting and analysis. Provide access to detail with out replicating across applications.
– Push downstream data to all systems
ERPi
ERPi
FDMEE - Key Features
FDMEE - Mapping
RDBMS JDBC – Find Out What it Means to Me
ConectionString="DRIVER=oracle.jdbc.OracleDriver;URL=jdbc:oracle:thin:@<mc-name>:<port-no>:<DbName>;uid=<uid>;pwd=<pwd>"
set con = CreateObject("ADODB.Connection")con.open(ConectionString) set RS = CreateObject("ADODB.Recordset")RS.Open "Select * FROM EBS", con, 3, 3
HEY!!!!! I can do anything with VBScript
Lifecycle Management (LCM)– Install Shared Services and the product and the LCM
functionality is available without any additional services– Execute lights-out processing from the command line to
integrate with third-party schedulers– Navigate through a migration using the Migration
Wizard– Set up a migration definition file that can be reused and
scheduled– Review one of several reports - Migration Status,
Comparison, and Audit - to manage your migrations
DRM— Data Relationship Management (formerly MDM)
— Enterprise master data management solution
— Source/target agnostic
— Synchronize, blend and compare hierarchies
— Customizable scripting and business rule validations
— Enforce enterprise master data policies
— Change tracking
— Roll back to point-in-time Hierachies Only?Wrong!
Internal Stakeholders
Culture
Stewardship Program
Delivery MechanismsBackend Applications
Infrastructure & Technology
DRM Governance, Stewardship & Change Management
Technology, Infrastructure & Delivery
Culture and Alignment
Constituency Analysis
Shared Services
External Stakeholders
Governance Program
Management
AlignmentDRM
DRM
• Very robust metadata management tool• Can manage and deploy data for Hyperion and
non-Hyperion systems• Does not process data• Formerly known as MDM (Razza)
DRM
DRM• Use Cases
– Hierarchies for dimensions in cube(s) are sourced from external system
– Multiple cubes need same hierarchies– Keep all Hyperion products in sync– Centralize and reduce metadata maintenance– Segregate metadata management from system
administration– FDM Mappings, ANY Mappings!
DRM - API
DRM -> EPMA Integration
Export Profiles display the Hierarchies that are to be exported based on the Dimension Type identified via Hierarchy Group naming
Custom designed System Exports are created as part of the Application Template loading. These Export profiles allow exporting of DRM Versions and Hierarchies into EPMA and specific EPM Application Views
New Fields added as part of the EPMA Integration to specify Target EPMA / EPM Application and its Dimension Type
Introducing Data Relationship Governance
• Governance Work list• Two types of workflows:
– Change requests– Remediation requests
• Single & multi-item requests• Pre-validate requests to
ensure quality on data entry• Calculate derived / inherited
attributes• Comments & attachments• Notifications & alerts• Request activity history
Enable change management and remediation workflows natively
Four steps to configuration
DRG Configuration
Workflow Model
Workflow Tasks
Workflow Stages
Workflow Node Access Groups
Model Filters
Essbase Integration Services
• Used to model cubes from relational data sources
• Can be used to build/update outline and load data
• Supports complex SQL if needed• Very fast if used properly
Essbase Integration Services
• Pros– Very flexible– Rich dimension building capabilities without
having to mess around with load rules• Cons
– Once model is setup, you’re “married to it”– You need to really think to get it right
Essbase Integration Services
Essbase Integration Services
• Use Case– Data and metadata already exist in a SQL
database, and now the business would like to build multiple cubes to provide analysis capabilities and possibly drill-thru
Essbase Integration Services
Essbase Studio
Essbase Studio• Used to design cubes from relational data
sources, OBIEE, EPMA apps & text file• Used for light-weight data modeling• Evolution of EIS• Common Metadata Repository• Lineage Tracking• Limited EAS functionality provided
Essbase Studio
Essbase Studio• Pros
– Very flexible– Enables true artifact reusability– Introspection is very cool– Drill through (RDBMS, URL, HFM, Java, etc.)
• Cons– Not your father’s Application Manager– Not your father’s EAS
Outline Load Utility• Can load data and metadata• Create a load file in text (CSV)• Complex command line optionsOutlineLoad [-f:passwordFile][/S:server] /A:application /U:userName [/M] [/E:outputFileName] [/I:inputFileName/D[U]:loadDimensionName|/DA:attributeDimensionName:baseDimensionName] [/TR] [/N] [[/R] [/U]] [/C] [/F] [/K] [/X:exceptionFileName] [L:logFileName] [/DX:HSP_Rates][/DS:HSP_SMARTLISTS] [/8][/?]
Outline Load Utility• Pros
– No ETL required– Perform batch outline updates to Planning– Can load data as well
• Cons– Arcane syntax– Practical limitations (Volume)
Outline Load Utility
Outline Load Utility
• Use Case– Company uses Planning to do yearly forecast of
employee salaries. HR provides a spreadsheet with employee data and salary basis data
– Outline load utility can be used periodically to update the employee members in the Planning outline and load up their base data
• Zero Level• Upper Level• All Data• Very fast• Column Format
Essbase Native Exports
• Easy to Use• Fairly Flexible• Can build data & Metadata• Very Modest “ETL”• Can be fast
Essbase Load Rules
@DATAEXPORT• Historically, an Essbase CDF (custom defined
function) used to export data to text file or relational data source
• Is now native functionality (no need to setup CDF, which can be tricky)
• Useful for calling from MaxL (in place of report script)
• Can provide export capability to users by giving them access to a calc script
@DATAEXPORT
• Pros– Can be easier and more flexible than report scripts
• Cons– Can only export data (no metadata)
• Use Case– Exporting data to be used to load another cube
@DATAEXPORT
• Use Case– Data exists in cube but not necessarily easily
accessible from other SQL sources, and wish to generate a text file with a certain subset of data (addressable with FIX)
@DATAEXPORT
@XREF | @XWRITE/*Enables a database calculation to write values to another Essbase database, or to the same database*/FIX (East, Actual, Budget, Sales) "100-30" ( @XWRITE("100-30", sam2basic, "100-20"); ) ENDFIX
/* Enables a database calculation to incorporate values from another Essbase database */Units = Units * @XREF(InflatDB,Inflation,US);
HFM Extended AnalyticsExtended Analytics enables you to export data and metadata from Financial Management applications to a flat file. The data and metadata are exported in a star schema format. Each star schema consists of a fact table and a number of dimension tables. Each record in the fact table of the star schema represents a cell or dimension intersection in the Financial Management data structure.
Essbase APIDeclare Function EsbLoadDatabase Lib "ESBAPIN" (ByVal hCtx As Long, ByVal AppName As String, ByVal DbName As String) As Long
Sub ESB_LoadDatabase () Dim sts As Long Dim AppName As String Dim DbName As String AppName = "Sample" DbName = "Basic" '************** ' Load Database '************** sts = EsbLoadDatabase (hCtx, AppName, DbName)
End Sub
MaxL
• Used to automate Essbase administrative tasks
• Replaced harder to read “ESSCMD”• Can do almost everything to a
server/application/cube/user that can be done in EAS
MaxL
• Commonly used to automate repetitive tasks and processes on an Essbase server
• Comes with Essbase (“free”)• Can import data (load rule), export data
(report script, calc)
ASO MDX Allocations
MaxLset MY_NAME = "ProcessCube";login $ESSUSER identified by $ESSPW on $ESSSERVER; spool stdout on to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.stdout.txt";spool stderr on to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.stderr.txt"; alter database $1.$2 reset data; import database $1.$2 data
connect as $SQLUSER identified by $SQLPW using server rules_file 'LdQVSOS'
on error write to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.error.txt"; execute aggregate process on database $1.$2
stopping when total_size exceeds $3; spool off;logout;exit;
MaxL
• Use Case– Data/metadata available in text file or relational
data source, want to automate on the server or work in to existing automation
– Perl Module
Batch Scripting
Batch Scripting
• Simple batch files used to perform a sequence of tasks
• Lends itself very easily to automating and scheduling for recurring tasks
• Can glue together other data integration tools: MaxL, ODI, outline load, etc.
Batch Scripting• Pros
– Very easy to implement– Not limited to Hyperion products
• Cons– Large scripts can become unwieldy and hard to
test– Many scripts are not good at dealing with an error
in the middle of processing– Custom code not supported– People my laugh at you
Batch Scripting
• Use case– Weekly close process consists of a series of
repetitive steps• Generate text file from another system• Copy text file• Load data to Essbase• Calc cube
– A simple batch file and MaxL script can be created and scheduled to automate this
Essbase SQL Loader• Used as the data source in a load rule• Primarily used for data loading• Also useful for dimension building• Very useful if you want to “skip” putting data
in a text file• Can serve as a stepping stone to using EIS,
ODI, Studio, etc.• Useful in conjunction with automation
EAL for HFM
HFMUser
EssbaseUser
EAL for HFM
EAL HEA
Extracts Data Directly to Essbase Yes No
Extracts Data to Relational Tables Yes Yes
Extracts Metadata to Relational Tables Yes Yes
Extracts Metadata Directly to Essbase Yes No
Does not require Data Manipulation Yes No
Does not require Data Load Rules Yes No
Does not require AIS Yes No
Real Time Data Synchronization Yes No
Manage Process from Essbase Yes No
EAL vs. Extended Analytics
EAL vs. OFMAEAL OFMA
Extracts Data Directly to Essbase Yes No
Extracts Data to Relational Tables Yes No
Extracts Metadata to Relational Tables Yes No
Extracts Metadata Directly to Essbase Yes No
Does not require Data Manipulation Yes Yes
Does not require Data Load Rules Yes Yes
Does require OBIEE No Yes
Real Time Data Synchronization Yes Yes
Manage Process from Essbase Yes No
Automatically builds OBIEE .rpd file No Yes
Automatically builds OBIEE Executive Dashboards No Yes
MediatorMain
Orchestration Composite
HumanWorkflow
Close Manager
Close Web Application Server (ADF)
Mediator
Synchronous Service
EventAdapters
End User Link
SOA
External System
Asynchronous Service
FCM – Data Flow (Integrated)
• System Automated tasks call an external webservice to perform their action– Webservice must support asynchronous calls– WSDL should contain separate porttypes for request and
response– Web service accepts WS-Addressing based reply information.
The WS-Addressing headers should be used by the Web service to direct responses to the correct callback service
– A client proxy is generated dynamically based on the definition– Any parameters are passed to the webservice using the
parameter code/value pair
FCM – Web Service(s) Integration
Account Reconciliation Manager (ARM)