ssis custom components

34
SSIS Custom Components Dave Ballantyne [email protected] @davebally

Upload: cale

Post on 14-Feb-2016

60 views

Category:

Documents


2 download

DESCRIPTION

SSIS Custom Components. Dave Ballantyne [email protected] @ davebally. Why ?. Provide new functionality not provided as standard. Why ?. Reusability Component is a DLL Single code base Can be used multiple times in a single project Can be shared across multiple projects - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: SSIS Custom Components

SSIS Custom Components

Dave [email protected]

@davebally

Page 2: SSIS Custom Components

Why ?• Provide new functionality not provided as

standard

Page 3: SSIS Custom Components

Why ?• Reusability

o Component is a DLLo Single code baseo Can be used multiple times in a single projecto Can be shared across multiple projectso Easy to test Component version

• Performanceo Faster than scripting

• Well documentedo Though not a how-to guide

Page 4: SSIS Custom Components

Types Of Component• Data Connections• Log Providers• For Each Loops• Control Flow Tasks• Data Flow Pipeline Component• Custom User Interface

Page 5: SSIS Custom Components

Pipeline Component Types

• Sources• Transforms• Destinations

Page 6: SSIS Custom Components

Design/Run Time• Design Time

o Work done in BIDSo attachments / detachmentso Validationo Column usage

• Run Timeo Metadata interrogationo DTEXECo Flow of data

Page 7: SSIS Custom Components

Demo 1• Reuse and Performance

Page 8: SSIS Custom Components

Performance Comparison

100000 500000 1000000 5000000 10000000 20000000 319996800%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

ScriptCustom

100,000

500,000

1,000,000

5,000,000

10,000,000

20,000,000

31,999,680

Custom 261 900 1,667 7,867 16,375 32,852 51,426Script 684 2,069 3,949 19,214 38,690 76,755 123,243

Page 9: SSIS Custom Components

Requirements• Visual Studio – BIDS is not enough• Or Visual Basic / C# Express• Client Tools SDK

Page 10: SSIS Custom Components

Starting Out• Target Framework 3.5 (Advanced compile options)• Sign the assembley• Add References(Program

file(x86)/<SqlServer>/100/sdk/Assemblies)o Microsoft.SqlServer.DTSPipelineWrapo Microsoft.SqlServer.DTSRuntimeWrapo Microsoft.SqlServer.ManagedDTSo Microsoft.SqlServer.PipeLine Host

Page 11: SSIS Custom Components

Class Creation• Inherits PipelineComponent• Uses attribute DtsPipelineComponent

Page 12: SSIS Custom Components

Post Build• Copy DLL to “C:\Program Files (x86)\Microsoft SQL

Server\100\DTS\PipelineComponents” • Register to Global Assembley Cache using

GACUTIL• Must Restart BIDS• For first use “Choose Items”,”SSIS Data Flow

Components” ,tick Component

Page 13: SSIS Custom Components

MetaData• IDTSComponentMetaData100• PipelineComponent.ComponentMetaData• Describes the Component to the engine• Inputs, Outputs• Custom data held within IDTSCustomProperty100

o Most level s of object

Page 14: SSIS Custom Components

MetaData• Inputs – IDTSInput100

o Exposed via InputCollection member in MetaDatao One instance for each attached inputo Contains virtual column collection

• Accessed with GetVirtualInput() member• View of the IDTSOutput100 of the Upstream component• IDTSVirtualInputColumn100

o Input Column Collection• Accessed with InputColumnCollection• Those that are used in the component• IDTSInputColumn100

o SetUsageType used to add the virtual column to the input column

Page 15: SSIS Custom Components

MetaData• Outputs – IDTSOutput100

o Exposed via OutputCollection member in MetaDatao One class for each outputo output Column Collection

• Accessed with OutputColumnCollection• IDTSOutputColumn100

• Dispositions – Errorso Set IsErrorOut on IDTSOutput100

Page 16: SSIS Custom Components

Icons• Size

• 16*16 For ToolBox• 32*32 For Design Surface• Order of “IconResources” is important

• Build action must be “Embedded Resource”

Page 17: SSIS Custom Components

Errors and warnings• FireError

o At design or run time

Page 18: SSIS Custom Components

Errors and warnings• FireWarning

Page 19: SSIS Custom Components

Design Time Methods• Methods

o ProvideComponentProperties• Define initial metadata of component

o Validate• Tests the metadata is correct

o ReinitializeMetaData• Fix the metadata

Page 20: SSIS Custom Components

Debug

Page 21: SSIS Custom Components

Demo 2• Build a simple component

Page 22: SSIS Custom Components

Run-Time Processing

Pre-Execute

PrimeOutput

ProcessInput

PostExecute

Page 23: SSIS Custom Components

PreExecute• Setup the runtime objects• Interrogate the Metadata and buffer manager• Find the colindex(s) in buffers based on metadata

o BufferManager.FindColumnByLineageID(InputId,InputCol.LineageId)

PrimeOutput

Page 24: SSIS Custom Components

Process Input• Loop on buffer.NextRow• If buffer.EndOfRowset is true set

outputBuffer.SetEndOfRowset()• MetaData functions are

not optimized for performance.

Page 25: SSIS Custom Components

PipelineBuffer• Used for both input and output buffer• Get<DataType> and Set <DataType>

o SetString / GetStringo SetInt32 / GetInt32

• AddRowo Insert and move to new row

• SetEndOfRowseto After final row has been poplulated

Page 26: SSIS Custom Components

Sync Or Async ?• Sync

o Add columns to existing data flowo SynchronousInputID of output = ID of input

• Asynco Create new data flow buffero SynchronousInputID =0

Page 27: SSIS Custom Components

Demo 3• RunTime execution

Page 28: SSIS Custom Components

User Interface

Page 29: SSIS Custom Components

User Interface

Page 30: SSIS Custom Components

User Interface• A Class that implements IDtsComponentUI• Registered to the component class with

UITypeName

• PublicKeyToken is found with GACUTIL

Page 31: SSIS Custom Components

User Interface

Page 32: SSIS Custom Components

User Interface• Demo 4

o User interface o UI Code Step Through

Page 33: SSIS Custom Components

Conclusion• Like SSIS , large learning curve• Reusability • Potentially Faster ?• .Net skills are required

Page 34: SSIS Custom Components

SSIS Custom Components

Dave [email protected]

@davebally