dat 332 sql server 2000 data transformation services (dts) best practices euan garden product unit...
TRANSCRIPT
![Page 1: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/1.jpg)
DAT 332
SQL Server 2000 Data Transformation Services (DTS) Best PracticesEuan GardenProduct Unit ManagerSQL Server DevelopmentMicrosoft Corporation
![Page 2: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/2.jpg)
Technologies Covered
Data Transformation Services
Analysis Services (Decision Support Objects)
Transact SQL
ADO
ActiveX Scripting
![Page 3: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/3.jpg)
“Best Practices” Agenda
Package Design
Data Loading
Advanced Techniques
Auditing and Error Handling
Analysis Services Management
![Page 4: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/4.jpg)
Package Design Practices
Drive Packages through MetadataDynamic Properties Task
Access to Property of any object
Sources values from many locales
Global VariablesBuilding blocks for communicating metadata
Populated by the Dynamic Properties Task
UDL FilesConfigurable File for storing Connection Information
![Page 5: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/5.jpg)
Package Design Practices
Designing Modular Parent/Child Packages
Execute Package TaskAllows joining of Transaction
Supports passing of Globals
Execution through COM AutomationProgrammatic access to Package objects
Eliminates Version Affinity
![Page 6: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/6.jpg)
Parent Child Packages
Child Package is Invoked by ActiveX Script
Can also be done with Execute Package Task
DTS Parent
DTS Child – Error Handler Example
![Page 7: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/7.jpg)
DemoDemo
Parent/Child Package(s) And UDL Files
Design Practices
![Page 8: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/8.jpg)
“Best Practices” Agenda
Package Design
Data Loading
Advanced Techniques
Auditing and Error Handling
Analysis Services Management
![Page 9: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/9.jpg)
Data Loading Practices
Leverage Fast Load Interface(s)Bulk Insert (Not always possible)
Data Pump (For More Complex Logic)
Characteristics of this ApproachBulk Insert Task
Execute SQL Task Rowset Parameter
ADO Disconnected Recordsets
DTS Multiphase Data Pump
![Page 10: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/10.jpg)
Data Loading Practices
Benefits to this ApproachCache Rowset on Client (fast lookups)
Hold updates on Client
Conditional Inserts/Updates without Data Driven Query
![Page 11: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/11.jpg)
DemoDemo
Conditional Inserts And Updates With The Multiphase Data Pump
Design Loading
![Page 12: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/12.jpg)
“Best Practices” Agenda
Package Design
Data Loading
Advanced Techniques
Auditing and Error Handling
Analysis Services Management
![Page 13: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/13.jpg)
Advanced Practices
Multi-Threaded Data LoadingRequires Multiple Connections
Data is Loaded in Parallel
Avoid Contention by Loading into Separate Entities
![Page 14: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/14.jpg)
Advanced Practices
Task LoopingRequires at least two tasks (Executive and Trigger)
Facilitated by Manipulating Workflow
<Step>.ExecutionStatus = DTSStepExecStat_Waiting
Maintain State in Global Variables
![Page 15: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/15.jpg)
Advanced Practices
Communication through Global Variables
Only Means for Communicating among Tasks
ExamplesPointer to Position within Global Array
Status of a Stored Procedure Call
Flag to Shutdown Parallel Load Processes
![Page 16: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/16.jpg)
Advanced Practices – Benefits
Higher Utilization during Data Load Compressed Data Loading Window Easy Orchestration within DTS
![Page 17: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/17.jpg)
Multi-Threaded Loading
Task MarshallTask Marshall
Thread 1 Thread 1 Loading TaskLoading Task
Initialization TaskInitialization Task
Thread n Thread n Loading TaskLoading Task
Thread 1 Thread 1 Restart TaskRestart Task
Thread n Thread n Restart TaskRestart Task
![Page 18: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/18.jpg)
DemoDemo
Multi-Threaded Data File Loading
Advanced Practices
![Page 19: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/19.jpg)
“Best Practices” Agenda
Package Design
Data Loading
Advanced Techniques
Auditing and Error Handling
Analysis Services Management
![Page 20: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/20.jpg)
Auditing And Error Handling
Logging OptionsLogging to MSDB
Logging to Text File
Logging to Event Log
Enabled through Package PropertiesCapture Step/Package Errors
Enabled through Data Pump PropertiesCapture Erroneous Data
![Page 21: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/21.jpg)
Auditing And Error Handling
DTSPackageLog Object WriteStringToLog()
WriteTaskRecord()
DTS Lineage Global VariablesDTSLineage_Full
DTSLineage_Short
DTSErrorRecords ObjectAdd()
Clear()
![Page 22: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/22.jpg)
Auditing And Error Handling
Monitoring Execution – Simple as Writing SQL
select
package.name, package.description, step.stepname,
step.errorcode, step.errordescription
from
msdb..sysdtspackagelog package, msdb..sysdtssteplog step
where
package.lineagefull = step.lineagefull and
step.errorcode <> 0
![Page 23: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/23.jpg)
DemoDemo
Custom Logging Using Intrinsic DTS Options
Error Handling
![Page 24: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/24.jpg)
“Best Practices” Agenda
Package Design
Data Loading
Advanced Techniques
Auditing and Error Handling
Analysis Services Management
![Page 25: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/25.jpg)
Analysis Services And DTS
Analysis Services Processing TaskSupports Processing of all OLAP Objects
Customizable Processing through TreeKey and other properties
Prominent PropertiesTreeKey ServerName\DBName\CubeFolder\CubeName
ItemType 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension
ProcessingOption 0=Full Process, 1=Refresh Data and 2=Incremental Update
![Page 26: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/26.jpg)
Analysis Services And DTS
Decision Support ObjectsProvide access to the AS Management Interfaces
Automated via AxScript
Supports Partition ManagementCreating
Cloning
Dropping
Merging of partitions
Should be an OLAP Administrator to Use
![Page 27: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/27.jpg)
DSO Hierarchy
![Page 28: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/28.jpg)
Analysis Services And DTS
BenefitsOpportunity to Dynamically Manage Partitions and other Objects
Support for Dynamic Processing
Inclusion in the BI Solution Load
![Page 29: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/29.jpg)
DemoDemo
Dynamic Partition Creation And Processing
Analysis Services
![Page 30: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/30.jpg)
Community ResourcesCommunity Resources
DTS Whitepaper – Search on MSDN for “Using DTS for Business Intelligence”
Microsoft Sitewww.microsoft.com/sql
SQLDTS Site www.sqldts.com
SQL Server Developers Network www.sqldev.net
SQLServer Centralwww.sqlservercentral.com
![Page 31: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/31.jpg)
Ask The ExpertsGet Your Questions Answered
I’ll be in the Experts booth right after this session
![Page 33: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/33.jpg)
Community Resources
Community Resourceshttp://www.microsoft.com/communities/default.mspx
Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/
NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx
User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx
![Page 34: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/34.jpg)
evaluationsevaluations
![Page 35: DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation](https://reader037.vdocuments.site/reader037/viewer/2022103122/56649f435503460f94c64585/html5/thumbnails/35.jpg)
© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.