informatica training

Download Informatica Training

Post on 17-Dec-2015

23 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

sdsd

TRANSCRIPT

  • Informatica

    Center Of Excellence-Data Warehousing

  • AgendaOverview & ComponentsInformatica Server & Data MovementRepository Server & Repository ManagerDesignerTransformations used in InformaticaRe-usable Transformations & MappletsWorkflow Manager & Workflow MonitorPerformance Tuning & Troubleshooting

    Center Of Excellence-Data Warehousing

  • Overview & Components

    Center Of Excellence-Data Warehousing

  • Informatica Power Center Architecture

    Center Of Excellence-Data Warehousing

  • PowerCenter 8x ArchitecturePackaged ApplicationsStandards, Messaging, Web ServicesPackaged ApplicationsRelational/Flat FilesMainframe/ MidrangeSourcesTargetsIntegration Service(s)PowerCenterConnectsPowerExchangeCore ServicesApplication ServicesClient ToolsSAP BW ServiceWeb Services HubRepository Service(s)Domain/Gateway ServiceAuthenticationConfigurationService management-

    Center Of Excellence-Data Warehousing

  • What is a Domain?Unified and single point of admin/config for:Integration Service (Informatica Server)Repository Service (Repository Server)Web Services Hub Service (WSH)BW Integration Service (BW Integration )Domain Consists ofSet of NodesSet of ServicesZero or more GridsSet of Resources

    Center Of Excellence-Data Warehousing

  • Gateway (Domain Controller) NodePurpose of the Gateway nodeStarts up and manages services running on the domainManages Configuration MetadataProvides Service lookup for clientsChecks for service availability via heartbeatsCoordinates failover of services HA for Gateway NodeOne or more nodes can be designated as Gateway nodesOnly one Master gateway node active at a timeElection process determines new Master

    Center Of Excellence-Data Warehousing

  • ServicesApplication ServiceService that is configured by the end user and represents a key visible component (Integration Service, Repository Service etc)External clients directly interact with these servicesCore ServiceInfrastructure (internal) service (Gateway Service, Logging Service etc)

    Center Of Excellence-Data Warehousing

  • HA SetupOne Primary node and list of Backup nodes (Active/Passive mode)Application ServicesCore ServicesAutomatic failover from Primary to BackupNo automatic fail-backManual fail-backIntegration Service operates in Active-Active mode

    Center Of Excellence-Data Warehousing

  • Overview .. Informatica RepositoryStores the metadata created using the Informatica Client toolsRepository Manager creates the metadata tables in the databaseTasks in the Informatica Client application such as creating users, analyzing sources, developing mappings or mapplets, or creating sessions creates metadataInformatica Server reads metadata created in the Client application when a session runsGlobal and local repositories can be created to share metadata

    Center Of Excellence-Data Warehousing

  • GridCollection on nodesIntegration Service can be assigned to GridService runs on all nodes in the gridGrid leveraged for Workflow distribution and Session distribution (SonG)ScalabilityAvailabilityAdvanced Load BalancerResource map

    Center Of Excellence-Data Warehousing

  • PowerCenter Architecture:Data FlowUNIX (AIX, HPUX, Solaris, Tru64) Windows NT 4.0, 2000Oracle MS SQL Server Sybase DB2 UDB Client Tools Windows 95, 98, NT 4.0 or 2000ObjectRepositoryRepository ServiceSourcesTargets

    Center Of Excellence-Data Warehousing

  • Overview .. Informatica Client ToolsRepository Manager To create and administer the metadata repositoryTo create repository users and groups, assign privileges and permissionsManage folders and locksDesigner To add source and target definitions to the repositoryTo create mappings that contain data transformation instructionsWorkflow Manager & Workflow MonitorTo create, schedule, execute, and monitor sessions

    Center Of Excellence-Data Warehousing

  • Overview .. Informatica ServerThe Informatica Server reads mapping and session information from the repositoryIt extracts data from the mapping sources and stores the data in memory while it applies the transformation rules in the mappingThe Informatica Server loads the transformed data into the mapping targetsPlatformsWindows NT/2000 UNIX

    Center Of Excellence-Data Warehousing

  • Overview .. SourcesRelational - Oracle, Sybase, Informix, IBM DB2, Microsoft SQL Server, and TeradataFile - Fixed and delimited flat file, COBOL file, and XMLExtended PowerConnect products for PeopleSoft, SAP R/3, Siebel, and IBM MQSeriesMainframe PowerConnect for IBM DB2 on MVSOther - Microsoft Excel and Access

    Center Of Excellence-Data Warehousing

  • Overview .. TargetsRelational - Oracle, Sybase, Sybase IQ, Informix, IBM DB2, Microsoft SQL Server, and TeradataFile - Fixed and delimited flat files and XML Extended Integration server to load data into SAP BW. PowerConnect for IBM MQSeries to load data into IBM MQSeries message queuesOther - Microsoft AccessODBC or native drivers, FTP, or external loaders

    Center Of Excellence-Data Warehousing

  • Questions

    Center Of Excellence-Data Warehousing

  • Informatica Server & Data Movement

    Center Of Excellence-Data Warehousing

  • Informatica Server and Data MovementThe Informatica Server moves data from sources to targets based on mapping and session metadata stored in a repository databaseA session is a set of instructions that describes how and when to move data from sources to targetsWorkflow Manager creates and manages and executes sessions,worklets and workflows.Workflow Monitor is used to monitor session for debugging in case of any error

    Center Of Excellence-Data Warehousing

  • Informatica ServerWhen a session starts, the Informatica Server retrieves mapping and session metadata from the repository database through Repository Server initiating a Repository Agent

    The Informatica Server runs as a daemon on UNIX and as a service on Windows NT/2000

    The Informatica Server uses the following processes to run a session: The Load Manager process - Starts the session, creates the DTM process, and sends post-session email when the session completesThe DTM process - Creates threads to initialize the session, read, write, and transform data, and handle pre- and post-session operations

    Center Of Excellence-Data Warehousing

  • The Load Manager Process The Load Manager performs the following tasks: Manages session,worklet and workflow schedulingLocks the session and reads session propertiesReads the parameter fileExpands the server and session variables and parametersVerifies permissions and privilegesValidates source and target code pagesCreates the session log fileCreates the Data Transformation Manager (DTM) process, which executes the session

    Center Of Excellence-Data Warehousing

  • The Load Manager ProcessThe Load Manager and repository communicate with each other using Unicode

    To prevent loss of information during data transfer, the Informatica Server and repository require compatible code pages

    It communicates with the repository in the following situations: When you start the Informatica ServerWhen you configure a sessionWhen a session starts

    Center Of Excellence-Data Warehousing

  • Data Transformation Manager ProcessDTM process is the second process associated with a session runThe primary purpose of the DTM process is to create and manage threads that carry out the session tasksThe DTM allocates process memory for the session and divides it into buffers. This is also known as buffer memoryIt creates the main thread, which is called the master threadThe master thread creates and manages all other threadsIf you partition a session, the DTM creates a set of threads for each partition to allow concurrent processingWhen the Informatica Server writes messages to the session log, it includes the thread type and thread ID

    Center Of Excellence-Data Warehousing

  • DTM Threads

    Center Of Excellence-Data Warehousing

    Thread Type

    Description

    Master Thread

    Main thread of the DTM process. Creates and manages all other threads. Handles stop and abort requests from the Load Manager.

    Mapping Thread

    One thread for each session. Fetches session and mapping information. Compiles the mapping. Cleans up after session execution.

    Pre- and Post-Session Threads

    One thread each to perform pre- and post-session operations.

    Reader Thread

    One thread for each partition for each source pipeline. Reads sources. Relational sources use relational threads, and file sources use file threads.

    Writer Thread

    One thread for each partition, if a target exists in the source pipeline. Writes to targets.

    Transformation Thread

    One or more transformation threads for each partition.

  • DTM ThreadsFor example, a pipeline contains one source, one target. You configure two partitions in the session properties. The DTM creates the following threads to process the pipeline: Two reader threads - One for each partition. Two writer threads - One for each partition

    When the pipeline contains an Aggregator or Rank transformation, the DTM creates one additional set of threads for each Aggregator or Rank transformation

    Center Of Excellence-Data Warehousing

  • DTM ThreadsWhen the Informatica Server processes a mapping with a Joiner transformation, it first reads the master source and builds caches based on the master rowsThe Informatica Server then reads the detail source and processes the transformation based on the detail source data and the cache dataThe pipeline for the master source ends at the Joiner transformation