Enterprise Data Integration Strategies for Microsoft Dynamics

CRM

Jean-René RoySalim Adamon

Who is Jean-Rene Roy?

• Independent Consultant• SQL Server MVP• Software Architect and team lead for many enterprises• Program Lead for OttawaSQL.Net and .NET Ottawa

Community• Founder of SQLTeach and DevTeach• SQL Server / CRM Dev / TFS Specialist• System Architect for TBS, HoC, CCA

Who is Salim Adamon?

• CRM Consultant @ “A Hundred Answers”• CRM and Technology Solution Architect• Microsoft Dynamics CRM MVP• CRM Blogger

Agenda

• Introduction to Dynamics CRM

• Data Migration vs Data Integration

• Data Migration & Integration Strategies

Examples and Challenges

• Pitfalls of developing with Dynamics CRM

What is Dynamics CRM?

• Dynamics CRM is a COTS product• Flexible Client Relationship Management platform

Allows for complex configuration Provides deep customization framework

• Customizations are limited by the CRM framework

Live by the FrameworkOr

Die by the Framework

CRM vs T-SQL

• Entity = DB Table• Entity Fields = Table Fields• Relationship 1 - N, N- N, N-1 = SQL Server Index• Bulk Edit = T-SQL Update statement • Workflow ~ T-SQL Triggers

Introduction to the CRM Database

DEMO

Introduction to the CRM Database

• Let’s take a look at CRM Database• What’s the type of the Primary key in Tables?• What’s a Dynamic View?• What is the Lookup Concept? (Advanced Find)

Introduction to the GUID Concept in CRM

Data Migration & Data Integration

Data migration/integration Very complex (diversity of data, integrity, downtime) Forgotten about (overlooked, treated as less important)

User adoption is driven by the quality of your data!

Data Migration vs Data IntegrationData Migration

• “One and done” activity

• Initial data load

• Moving large volume of data

• Get it right

• Bring in only relevant data

• Cost to fix data issues is high

Data Integration

• On-going replication or

synchronization of data

• Different needs:

• Real Time

• Batch (scheduled based)

• On-demand (messaging/queue)

• Needs to be maintained as the

application evolves

Data Migration / Integration Strategies

• CRM Data Import Wizard

• ETL Tools SSIS + 3rd Party CRM Connector

Scribe

Connectors for MS Dynamics

More…

• Custom Integration Development Write your own using CRM web services SDK or service reference

• BizTalk / MSMQ (Message queue)

CRM Data Import Wizard

• Using XML/CSV Templates• Configure mapping and Import data into CRM

DEMO

Data Migration/Integration Strategies

ETL Tools• Leverage SQL Server Integration Services (SSIS)

Performance Scalability Extensibility Scheduling capabilities

• Dynamics CRM Platform restrictions: Allows to read data from database using System View Create, Update, Delete operations NOT allowed at the database level Create, Update, Delete must be done via CRM SDK or web service reference

• Solution Use 3rd party vendor (SSIS Integration Toolkit with CRM Connectors) Kingsway Software

Data Migration/Integration Strategies

ETL Tools• Best practices (why doing it like this?)

Done at the beginning Run every day and adjusted Why syncing the GUID

DEMOGUID-Sync ETL

Data Migration/Integration Strategies

Need a break?

Data Migration/Integration Strategies

Back in 15 minutes…

ETL Tools• Continuous Data Integration

Introduce the Sync Data Concept Sync Data in one direction vs two directions Why do we do this?

Data Migration/Integration Strategies

ETL Tools : Batch Data Processing

• Talk about the User Story for Check Payment• Show what can be a Batch Integration• Why doing it in Batch?• Show the SQL Server Agent Calling SSIS

DEMOBatch Check SSIS Package

Data Migration/Integration Strategies

Custom Development

• Using .NET Code with XRM SDK or Service Reference CRUD operations Data Integration using multiple sources You are the ultimate decider!

• Why custom development? Real-time integration Keep all components as part of a single “CRM Solution” (no additional

components to deploy) Leverage internal resources strengths Learning curve and cost of external integration tools

Data Migration/Integration Strategies

Custom Development

• Real-time Integration using Plugins Custom Business Logic (code .NET/C# or VB) Event handles for events fired by Dynamics CRM

Example of events: Create, Update, Delete, Set State etc.

DEMO

Data Migration/Integration Strategies

Custom Development

• Other Possibilities Workflows

Asynchronous or Synchronous Configurable (screen editor) Functionality can be extended with Custom Code No execution timeout

Dialogs Interactive and synchronous process Requires user input to start and run to completion Functionality can be extended with Custom Code

Data Migration/Integration Strategies

Pitfalls of developing with Dynamics CRM

• Traditional Software Development : 25 Analyze 75% coding

75%

25%

25%

75%• Development with CRM :

75% Analyze 25% coding

Pitfalls of developing with Dynamics CRM

• Using CRM for high volume data: plan carefully

• Using COST product to build custom process can take a long time.

Need an adapted development process

• Involve users early (UAT/Training)

• Define source control strategies early

• Re-design existing process for the framework can be challenging Remodel your processes to best use the out of the box features

Questions?

Contact Info

• Jean-René Roy Email: jrroy@devteach.com Blog: http://www.softdesign.ca/blog/

• Salim Adamon Email: s.adamon@outlook.com Blog: http://thedynamicscrmblog.wordpress.com/

References• Third party:•  https://www.kingswaysoft.com•  https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-crm/purchase•  • Blogs / Forum:•  http://danielcai.blogspot.ca/2012/02/improve-crm-data-load-performance-by.html•  http://danielcai.blogspot.ca/•  http://petecrm2011.wordpress.com/page/2/•  http://guruprasadcrm.blogspot.com.au/2011/07/retrieving-optionset-lable-data-using.html•  http://stackoverflow.com/questions/10443253/how-to-set-option-set-value-in-crm-2011•  • SSIS BDD:• http://technet.microsoft.com/en-us/sqlserver/hh369962• http://www.microsoft.com/en-ca/download/details.aspx?id=4123•