![Page 1: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/1.jpg)
Taking Your Application Design To The Next Level With Data Mining
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Next Level With Data Mining
Peter Myers
Mentor – Solid Quality Mentors
Silicon Valley SQL Server User Group – 21 July, 2009
![Page 2: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/2.jpg)
•Peter Myers
•Mentor and Trainer, Solid Quality Mentors
•BBus, MCP, MCITP (DBA, Dev, BI), MCT, MVP
•12 years’ experience designing, developing and
PRESENTER
Copyright © 2009, Solid Quality Mentors. All rights reserved.
•12 years’ experience designing, developing and supporting software solutions using Microsoft data and development platforms
![Page 3: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/3.jpg)
• Industry experts:
Growing, elite group of over 90 of the world’s best technical experts who, as reflected by the high concentration of Microsoft MVP’s and RD’s in our ranks, achieve excellence in their industry by maintaining the highest credentials.
• Published authors:Best technical reference books, Microsoft reference materials, industry white
WHO WE ARE
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Best technical reference books, Microsoft reference materials, industry white papers, technical magazine articles, and webcasts.
• Top technical speakers:PASS Community Summit, Microsoft TechEd, The Microsoft BI Conference, SQL Server DevConnections, countless user groups, international conferences and events.
• For more information visit www.solidq.com
![Page 4: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/4.jpg)
Provide advanced, world-class expertise across the entire Microsoft relational data and development platforms and
complimenting technologies.
WHAT WE DO
PRACTICE AREAS SERVICES
Copyright © 2009, Solid Quality Mentors. All rights reserved.
For more information visit www.solidq.com
PRACTICE AREAS SERVICES
Relational Database Management Advanced, Public Training
Business Intelligence Customized, Private Training
Development Methodologies Solution Delivery & Tuning
SharePoint Collaboration Enhanced, Mentoring Services
![Page 5: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/5.jpg)
• Introducing Data Mining
•Describing the Data Mining Process
• SQL Server™ 2008 Data Mining
•Data Preparation
AGENDA
Copyright © 2009, Solid Quality Mentors. All rights reserved.
•Data Preparation
•Data Mining Visualization
•Demonstrations
![Page 6: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/6.jpg)
•Addresses the problem:“Too much data and not enough information”
• Enables data exploration, pattern discovery, and pattern prediction—which lead to knowledge discovery
INTRODUCINGDATA MINING
Copyright © 2009, Solid Quality Mentors. All rights reserved.
prediction—which lead to knowledge discovery
• Forms a key part of a BI solution
![Page 7: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/7.jpg)
DATA MININGENABLES PREDICTIVE ANALYSIS
Predictive Analysis
Proactive Data mining
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Presentation Exploration Discovery
Passive
Interactive
Business Insight
Canned reporting
Ad-hoc reporting
OLAP
![Page 8: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/8.jpg)
• Identifying responsive customers/unresponsive customers (also known as churn analysis)
• Targeting promotions
•Detecting and preventing fraud
BUSINESSSCENARIOS
Copyright © 2009, Solid Quality Mentors. All rights reserved.
•Detecting and preventing fraud
•Correcting data during ETL
• Forecasting sales and inventory
•Cross-selling
![Page 9: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/9.jpg)
DESCRIBING THEDATA MINING PROCESS
“Doing Data Mining”
Business Understanding
Data Understanding
Data Preparation
Copyright © 2009, Solid Quality Mentors. All rights reserved.
“Putting Data Mining to Work”
Data Preparation
Modeling
Evaluation
Deployment
Data
www.crisp-dm.org
![Page 10: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/10.jpg)
• Often significant amounts of effort are required to prepare data for mining:
• Transforming for cleaning and reformatting
• Isolating and flagging abnormal data
• Appropriately substituting missing values
DATAPREPARATION
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Appropriately substituting missing values
• Discretizing continuous values into ranges
• Normalizing values between 0 and 1
• Of course, having the required data to begin with is important:
• When designing systems, give consideration to attributes that may be required as inputs for classification
o For example, demographic data: Age, Gender, Region, etc
![Page 11: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/11.jpg)
MODELING
Design time
Process time
Query timeMining Model
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Mining Model
![Page 12: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/12.jpg)
MODELING
Design time
Process time
Query timeMining Model
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Mining Model
Training Data
Data Mining Engine
![Page 13: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/13.jpg)
MODELING
Design time
Process time
Query timeMining Model
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Data Mining Engine
Data to PredictPredicted Data
Mining Model
![Page 14: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/14.jpg)
• It is important that the model makes sense
• Accuracy
o Does it correlate and predict correctly?
• Reliability
MODELVALIDATION
Copyright © 2009, Solid Quality Mentors. All rights reserved.
o Does it work similarly for different test data?
• Usefulness
o Does it provide insight or only obvious trivialities?
•Commonly a holdout set of data is used to test model accuracy
![Page 15: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/15.jpg)
• Hides the complexity of an advanced technology
• Includes full suite of algorithms to automatically extract information from data
• Handles large volumes of data and complex data
•
SQL SERVER™ 2008 DATA MINING
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Data can be sourced from relational and OLAP databases
• Uses standard programming interfaces:
• XMLA
• DMX
• Delivers a complete framework for building and deploying intelligent applications
![Page 16: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/16.jpg)
INTEGRATEDEND-TO-END OFFERING
END USER TOOLS & PERFORMANCE MANAGEMENT APPS
ExcelExcel PerformancePoint ServerPerformancePoint Server
SharePoint ServerSharePoint Server
DELIVERY
ReportsReports DashboardsDashboards Excel Excel WorkbooksWorkbooks
AnalyticAnalyticViewsViews ScorecardsScorecards PlansPlans
Copyright © 2009, Solid Quality Mentors. All rights reserved.
ExcelExcel PerformancePoint ServerPerformancePoint Server
BI PLATFORM
SQL Server SQL Server Reporting ServicesReporting Services
SQL Server SQL Server Analysis ServicesAnalysis Services
SQL Server DBMSSQL Server DBMS
SQL Server Integration ServicesSQL Server Integration Services
![Page 17: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/17.jpg)
SQL SERVER™ 2008ALGORITHMS
• Microsoft Naïve Bayes
• Quick and approachable algorithm
• Used for classification
• Microsoft Decision Trees
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Microsoft Decision Trees
• Popular data mining technique
• Used for classification, regression and association
• Microsoft Linear Regression
• Finds the best possible straight line through a series of points
• Used for prediction analysis
![Page 18: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/18.jpg)
SQL SERVER™ 2008ALGORITHMS
• Microsoft Neural Network
• More sophisticated than Decision Trees and Naïve Bayes, this algorithm can explore extremely complex scenarios
• Used for classification and regression tasks
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Microsoft Logistic Regression
• A particular case of the Neural Network algorithm
• Microsoft Clustering
• Finds natural groupings inside data
• Supports segmentation and anomaly detection tasks
![Page 19: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/19.jpg)
SQL SERVER™ 2008ALGORITHMS
• Microsoft Sequence Clustering
• Groups a sequence of discrete events into natural groups based on similarity
• Microsoft Time Series
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Microsoft Time Series
• Used to predict future values from a time series
• Has been improved in SQL Server 2008 to produce more accurate long-term forecasts
• Microsoft Association Rules
• Commonly supports market basket analysis to learn what products are purchased together
![Page 20: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/20.jpg)
SQL SERVER™ 2008ALGORITHMS
Classify
• Decision Trees
Estimate
• Decision Trees
Cluster
• Clustering
Forecast
• Time Series
Associate
• Association Rules
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Logistic Regression
• Naïve Bayes
• Neural Networks
• Linear Regression
• Logistic Regression
• Neural Networks
• Decision Trees
![Page 21: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/21.jpg)
• In contrast to OLTP and OLAP queries, data mining queries typically extract information that the user is not aware of
• Appreciate that end users do not typically query data mining models directly
• Visualizations can effectively present data discoveries
DATA MININGVISUALIZATION
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Visualizations can effectively present data discoveries
• SQL Server™ 2008 provides algorithm-specific visualizations that can:
• Test and explore models in BIDS
• Be embedded into Web and Windows Forms applications
• Developers can construct and plug-in custom data mining viewers
![Page 22: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/22.jpg)
DATA MININGPROGRAMMABILITY
WANXMLAOver TCP/IP
OLE DB ADO ADOMD.NET
XMLA
Any Platform, Any Device
C++ App VB App .NET App Any App
AMO
Copyright © 2009, Solid Quality Mentors. All rights reserved.
Data Mining Interfaces
Analysis Server
OLAP Data Mining
Server ADOMD.NET
.NET Stored Procedures
Microsoft Algorithms
Third-Party Algorithms
XMLAOver HTTP
![Page 23: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/23.jpg)
•AMO (Analysis Management Objects)
• Administer database objects
• Apply security
• Manage processing
ANALYSIS SERVICESAPIs
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Manage processing
•ADOMD.NET
• Connect to SSAS databases
• Retrieve and manipulate data
• Server ADOMD.NET
• Extend DMX by using .NET stored procedures
![Page 24: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/24.jpg)
DEMONSTRATIONS
Copyright © 2009, Solid Quality Mentors. All rights reserved.Copyright © 2008, Solid Quality Mentors. All rights reserved.
1. Creating, Training, Testing and Querying Mining Models with BIDS 2. Embedding Visualizations Into a Windows Forms Application3. Embedding a Data Mining Report Into a Windows Forms Application 4. Enhancing an E-Commerce Site with Targeted Marketing5. Enhancing an E-Commerce Site with Market Basket Analysis6. Extending DMX With a .NET Stored Procedures7. Automating Data Validation With Data Mining
![Page 25: SQL Server Data Mining - Taking your Application Design to the Next Level](https://reader034.vdocuments.site/reader034/viewer/2022051515/54c771924a7959b6248b45f4/html5/thumbnails/25.jpg)
•www.microsoft.com/sql/technologies/dm
• Links to technical resources, case studies, news, and reviews
•www.sqlserverdatamining.com
• Site designed and maintained by the SQL Server Data Mining
RESOURCES
Copyright © 2009, Solid Quality Mentors. All rights reserved.
• Site designed and maintained by the SQL Server Data Mining team
• Includes: Live samples, tutorials, webcasts, tips and tricks, and FAQ
•Data Mining for SQL Server 2008, by ZhaoHui Tang and Jamie MacLennan