SQL Server 2012 Integration ServicesLevel 300

SQL Server 2012 Integration Service – Core concepts

SSIS Catalog

3

Configuration

Set values for parameters

Central connection manager configuration

Advanced property override functionality

Security

Encryption of projects and parameter values

Row-level security to control access to packages

Management

Interactive package execution and SQL Agent integration

Dashboard and built in reports for troubleshooting

SSIS Server Architecture

4

SQ

L S

erv

er

Insta

nce

SSIS Server

SSIS Catalog

Deploy Manage Security Validate Execute

IS Objects

Security

Operation logs

State

Execution Process

Execution Control

Components

PowerShell SSMS Deployment Wizard Application

OM: Manage OM: Manage OM: Manage OM: Project OM: Manage OM: Project

Runtime

The SSIS Catalog: SSISDB

5

• Provides a set of stored procedures and views for managing, configuring, executing and monitoring SSIS packages deployed to the SSIS Catalog

• The Catalog schema is designed for customer use, and is a supported API• Contains views, stored procedures, and functions

• The Internal schema is neither public nor supported• Contains base tables and lower-level objects

• All projects and sensitive values are encrypted • Protected by a pair of certificate and symmetric key• Utilizes the built-in SQL Server encryption

infrastructure

SSISDB

The SSIS Catalog: SSISDB (cont.)

6

• Whatever you do using SSMS can be scripted either using T-SQL or PowerShell

• Important: Backup the database master key• http://blogs.msdn.com/b/mattm/archive/2012/03/23/

ssis-catalog-backup-and-restore.aspx

SSISDB

SSISDB Overview

7

Tables

Views

SSISDB

Stored Procedures

Stored Procedures

SQLCLR Assembly

ISServerExec.exe

Microsoft.SqlServer.IntegrationServices.Server. Shared.dll

Functions

• 42 store procedures• 25 views• 3 functions• 1 trigger• 1 SQL CLR assembly

Public objects:

Internal CatalogTriggers

SSISDB: Usage Patterns

8

Client SQL Server instance

Entry point: T-SQL sp

InvokedCLRsp

Return success/failThrow if errors occur

External ProcessISServerExec

Client SQL Server instance

InvokedCLR SP

Return success/failThrow if errors

occur

Create process by impersonating

caller of the stored proc

Either Asynchronous or Synchronous

T-SQL sp (entry point)T-SQL sp invokes managed sp

T-SQL sp (entry point)T-SQL sp invokes managed spManaged sp creates external process by impersonating caller of sp

Client SQL Server instance

T-SQL sp

Return success/failThrow if errors occur

T-SQL spCASE A CASE B CASE C

Entry point: T-SQL sp

SSISDB - Protecting Sensitive Data

9

Guiding Design Principle: No sensitive data should be stored in plain text in the server

Service Master Key

Database Master key

Project CertificateEnvironment Certificate

Execution Certificate

Project Symmetric key

Environment Symmetric key

Execution Symmetric key

Project Parameter

values

Environment values

Execution Parameter

values

Supported 'TRIPLE_DES_3KEY', 'AES_128', 'AES_192', 'AES_256'T-SQL functions: EncryptByKey; DecryptByKeyNote: We do not support sensitive data with length > 8000

SET @key_name = 'MS_Enckey_Proj_'+CONVERT(varchar,@project_id) SET @certificate_name = 'MS_Cert_Proj_'+CONVERT(varchar,@project_id)OPEN SYMMETRIC KEY key_name DECRYPTION BY CERTIFICATE certificate_nameSELECT parameter_name, DECRYPTBYKEY([sensitive_parameter_value]) FROM internal.[object_parameter_values]WHERE [project_id] = @project_idCLOSE SYMMETRIC KEY key_name

Note: The Catalog Views will automatically decrypt the values for you.

What’s new in SQL Server 2012 Integration Service

INTRODUCTION

11

• 4th release of Integration Services (SSIS)

• Improvements and features:• Usability for new developers• Productivity for developers• Deployment, configuration and management• Other improvements

• Migration

Usability for new developers

• “Getting Started” window with links to samples and videos

• New SSIS Toolbox:• Exposes a description and links to samples for a selected component• Allows categorizing, including Favorites

• Data flow connection assistants to facilitate the configuration of sources and destinations

Improved productivity for developers

• Undo/Redo• Data flow Script component debugging• SQL Server Data Tools (SSDT) designer• Updated look and feel• New VSTA scripting environment with support

for .NET 4• Auto-save and recovery• Improved performance when opening packages • Project connection managers• Icon marker to indicate when expressions are used

Improved productivity for developers (cont.)

• Variable window improvements • Variables are created at the package scope by default• Button to allow changing variables scope

• New data flow column mapper• Automatic column mapping where name and data

type matches• Simplified data viewers• Simplified package format• Makes it easier to diff changes

Improved productivity for developers (cont.)

• Enhanced data flow components:• Pivot transform user interface• Row Count transform user interface

• New control flow tasks:• CDC Control task• Expression task

• New data flow components:• ODBC source and destination• CDC Source source• CDC Splitter transform• DQS Cleansing transform

Deployment, configuration and management

• SSIS catalog:• User database (SSISDB) hosted on a SQL relational instance

with a collection of views and stored procedures that provide a T-SQL API

• New project model for bundling project resources together to simplify deployment (.ispac)

• New parameter model to simplify configuration management• Similar to parameters in programming functions• Read-only variable in a special namespace

Deployment, configuration and management (cont.)• Available with the SSIS catalog:• Connection manager properties are automatically exposed

on the server• Environments can be created to run packages with different

settings• Automatic capture of package execution logs• Data tap functionality, to dynamically capture data as it

flows through the data flow, and without modifying the package

• Server can be managed using PowerShell

Improved deployment, configuration and management (cont.)• Available with the SSIS catalog (cont.):• Built-in reports:• Integration Services Dashboard – view all packages

that have run on the server in the past 24 hours• Performance Reports – view a package’s performance

over time, at package and component level• Error Message Report – details failed package

executions and related error messages• Support for custom reports

Other Improvements

• Annotations auto-grow, wrap when you press Enter, and are persisted as clear-text

• Expression improvements:• 4000 character limit is removed• New functions – REPLACENULL, LEFT, TOKEN, and TOKENCOUNT

• Flat file supports embedded qualifiers and a variable number of columns per row

• Raw files now include sort information• Improved error messages for the Execute SQL task• Option to allow automatic type conversion in the Execute

SQL task parameters and result values

Migration

• Projects developed with SQL Server 2005, 2008 and 2008 R2 can be converted

• Projects are converted by using the Integration Services Project Conversion Wizard, launched from:• SSDT (.dtproj and .dtsx files are upgraded)• SSMS (generates a project deployment file – .ispac )

• The wizard can:• Update Execute Package tasks• Replace package configurations with parameters

• Migration of Data Transformation Services (DTS) solutions is not supported

• All DTS components have been removed• DTS Migration Wizard• Execute DTS Package task• DTS support in SSMS

• ActiveX Script task has been removed

Upgrading to SSIS 2012

Resources

Microsoft Confidential 22

• 5 Tips for a Smooth SSIS Upgrade to SQL Server 2012• SSIS Catalog - Backup and Restore• Troubleshooting SSIS Package Performance Issues• SSIS Operational and Tuning Guide

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION