columnstore customer stories 2016 by sunil agarwal
TRANSCRIPT
Agenda
World Record Breaking Performance (TPC-H)
SQL Server Leading the TPC-H benchmark
SQL Server 2016: Columnstore ImprovementsPerformance
Functionality: Data Warehouse (DW)
Functionality: Real-Time Operational Analytics
World Record Breaking Performance (TPC-H)
SQL Server 2016 gives 40% improved
performance over SQL Server 2014
Customer – Daman (Health Insurance, UAE)Scenario• Traditional DW with ~100 target tables, ~200 source
tables several billion rows
• Multi-layered, holistic, dimensional model, used by all
users
• Queries involve multi-table joins
Customer – Daman Requirements/Challenges
Requirements
Challenges
Approach
JulJunMayAprMarFebJan
2015 2016
DecNov
Go: POC SQL Server 2016
POC SQL Server 2016 Go
Production
Go
In the cloud
6 contenders
Synthetic data
Same data model
Similar characteristics
On the premises
SQL Server 2016 vs.
Sybase IQ
Full productive data set Daily load on SQL Server 2016
Automated test framework
Full test end user tools
Customer – Daman: Benchmark Results
Solution - CCI
Results – testing & production
50 reports weekly
Scheduled Reports
60% improvement
SQL 2016: 10 h
Sybase IQ: 28 h
Weekly
Runs at EOM
Business Tool
5 x improvement
SQL 2016: 10h
Sybase IQ: 2 d
Monthly
100s with 30 concurrent users
AdHoc Analytics
Significant Improvement
Daily
Many queries < 20
seconds
Demo – Screen Shots
Customer Daman: Learnings
ETL
Optimizer
Case Studyhttps://customers.microsoft.com/en-us/story/daman
First American - Business Problem
• Scope of Data:150 Million Properties in USA
• Querying
• Not a traditional Columnstore Index Scenario
• Failed Solution
First American - Requirements
Success Criteria
Examples of Slow SQLs SELECT
FROM
WHERE
SELECT
FROM
WHERE
19
Relational Table (disk-based)(Clustered Index)
20
Btree index
Rowstore Table (disk-based)
Hybrid queries are directed appropriately
Challenge
Solution
First American - Hybrid Query
SELECT <Select List>FROM NormandySearch.Search.PropertyWHERE propertyid in(SELECT propertyid
FROM NormandySearch.Search.PropertyCSWHERE [NumericStreetNumber] = 416
and OwnerName like 'BOW%‘ And ( [City] = 'Bay Village'Or [PlaceName] = 'Bay Village')
First American - Results with Columnstore Index
Data Compression
CPU Consumption
Query Performance with 100 concurrent users
With Rowstore (PAGE Compressed) With CCI
Database Size (including Indexes) 560 GB 44 GB (13x)
First American - Learnings
FIS – Business Overview and Challenges
Application Overview
Current Solution
FIS – Application Challenges
Challenge
No Columnstore Index Benefits
What to do?
FIS – Solution
Considered two configurations
Baseline with all 1.2 million rows compressed
Delivered solution with NCCI with force compression
FIS – Performance Numbers
Status - Application Live in Production
FIS Case Study
what something is instead of where it's stored.
Organize information based on
Proposal
ESTT Corporation
9/30/2016
Website Renewal
Y:
Proposals 2016
Projects
ESTT
Official docs
Website renewal
M-FILES – Document Management
Each document inserted results in approximately 60 SQL inserts into Metadata
Metadata change in a
document can result
updating metadata of
large number of other
documents
Document loading and Viewing
Metadata Table
Btree Index
Query Before (sec) After (sec)
Contracts by Pricing Type and Agency 6.5 0.5
Contracts by year and month 2.6 0.05
Thank you
for your time!