relational engine enhancements james hamilton language processing & exec dev manager sql server...

48
Relational Engine Relational Engine Enhancements Enhancements James Hamilton James Hamilton Language Processing & Exec Dev Language Processing & Exec Dev Manager Manager SQL Server SQL Server Microsoft Corp Microsoft Corp [email protected] [email protected] 5-206 5-206

Upload: kristopher-johnson

Post on 26-Dec-2015

224 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Relational Engine Relational Engine EnhancementsEnhancements

James HamiltonJames Hamilton

Language Processing & Exec Dev ManagerLanguage Processing & Exec Dev Manager

SQL ServerSQL Server

Microsoft CorpMicrosoft Corp

[email protected]@microsoft.com

5-2065-206

Page 2: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206
Page 3: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Business OperationsBusiness OperationsAvailabilityAvailabilityScalabilityScalabilityPerformancePerformanceProgrammabilityProgrammabilityISV drivenISV drivenInternet/CommerceInternet/Commerce

Data WarehousingData WarehousingLarge DatabasesLarge DatabasesComplex QueriesComplex Queries

Closed Loop AnalysisClosed Loop AnalysisPartitionsPartitions

OLAPOLAP Data TransformationData Transformation

Knowledge ManagementKnowledge ManagementContent Indexing/SearchContent Indexing/Search

Content/Web ServerContent/Web ServerTrackingTrackingWorkflowWorkflow

CoreCoreReliabilityReliability

Auto Tuning Auto Tuning SecuritySecurity

Base StorageBase StorageReplicationReplicationMeta DataMeta DataDevicesDevices

Storage Market/RequirementsStorage Market/Requirements

Page 4: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Business OperationsBusiness OperationsRow-Level LockingRow-Level LockingMSCS Failover ClustersMSCS Failover ClustersPrepare/ExecutePrepare/ExecuteDynamic DDLDynamic DDLFFO CursorsFFO Cursors

Data WarehousingData WarehousingOLAP Services OLAP Services

Intra-query ParallelismIntra-query ParallelismAdvanced Join TechniquesAdvanced Join TechniquesStar-Schema OptimizationsStar-Schema Optimizations

Data TransformationData TransformationDW FrameworkDW Framework

Knowledge ManagementKnowledge ManagementFull-Text IndexingFull-Text Indexing

Access Workflow DesignerAccess Workflow Designer

CoreCoreReliabilityReliability

Dynamic MemoryDynamic MemoryAuto Statistics Auto Statistics

WinNT-based SecurityWinNT-based SecurityVLDB UtilitiesVLDB Utilities

New On-Disk StructureNew On-Disk StructureMerge ReplicationMerge ReplicationUnicode SupportUnicode SupportDistributed QueryDistributed Query

SQL ServerSQL Server™™ 7.0 7.0

Page 5: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 7.0 SQL Server 7.0 Best TCO in the industryBest TCO in the industry Most Complete DW offeringMost Complete DW offering Handle 95% of all ERP customers needsHandle 95% of all ERP customers needs

(SAP, Baan, Psoft…)(SAP, Baan, Psoft…) >50% share on Internet Shopping Sites>50% share on Internet Shopping Sites Largest database on the InternetLargest database on the Internet

http://terraserver.microsoft.comhttp://terraserver.microsoft.com Best SAP POS B-mark on ANY platformBest SAP POS B-mark on ANY platform Best TPC-C price/perf in IndustryBest TPC-C price/perf in Industry Best TPC-C per/processor perf in industryBest TPC-C per/processor perf in industry Achieving 99.9x% availability TODAY!!!Achieving 99.9x% availability TODAY!!!

(Pennzoil, Barnes & Noble, E-Plus, Terraserver, more…)(Pennzoil, Barnes & Noble, E-Plus, Terraserver, more…) Over 8.5M seats sold in FY ’99Over 8.5M seats sold in FY ’99

Page 6: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 7.0: SQL Server 7.0: Relational EngineRelational Engine State of the art QP for OLTP, DSSState of the art QP for OLTP, DSS Intra-Query ParallelismIntra-Query Parallelism Advanced join technologyAdvanced join technology Caching of query plansCaching of query plans UnicodeUnicode Distributed queriesDistributed queries Full-TextFull-Text Auto Management (memory, stats)Auto Management (memory, stats)

Page 7: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple instances/machineMultiple instances/machine SecuritySecurity

Scalability, performanceScalability, performance AvailabilityAvailability

Page 8: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

ProgrammabilityProgrammability

Cascaded DRICascaded DRI User Defined FunctionsUser Defined Functions Column Level CollationsColumn Level Collations Instead Of TriggersInstead Of Triggers New DatatypesNew Datatypes FullText EnhancementsFullText Enhancements Extended PropertiesExtended Properties ChecksumChecksum Session contextSession context

Page 9: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Cascaded DRICascaded DRI

Automatic cascading of Deletes Automatic cascading of Deletes andand Updates Updates from PK to FK tablesfrom PK to FK tables

ANSI Standard Restrict and Cascade semanticsANSI Standard Restrict and Cascade semantics

CREATE TABLE country ( CREATE TABLE country ( country_name NVARCHAR(75) NOT NULL PRIMARY KEY ) country_name NVARCHAR(75) NOT NULL PRIMARY KEY )

CREATE TABLE employee ( CREATE TABLE employee ( employee_name NVARCHAR(75) NOT NULL,employee_name NVARCHAR(75) NOT NULL, nationality NVARCHAR(75) NOT NULL REFERENCES country nationality NVARCHAR(75) NOT NULL REFERENCES country ON UPDATE CASCADE ON UPDATE CASCADE ON DELETE NO ACTION,ON DELETE NO ACTION, passport_number VARCHAR(25) NOT NULL, passport_number VARCHAR(25) NOT NULL, PRIMARY KEY (nationality, passport_number))PRIMARY KEY (nationality, passport_number))

CREATE TABLE renewal_reminder ( CREATE TABLE renewal_reminder ( nationality NVARCHAR(75) NOT NULL,nationality NVARCHAR(75) NOT NULL, passport_number VARCHAR(25) NOT NULL, passport_number VARCHAR(25) NOT NULL, date_of_expiry datetime NOT NULL,date_of_expiry datetime NOT NULL, FOREIGN KEY (nationality, passport_number) REFERENCES employeeFOREIGN KEY (nationality, passport_number) REFERENCES employee ON UPDATE CASCADEON UPDATE CASCADE ON DELETE CASCADE )ON DELETE CASCADE )

Page 10: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Shiloh: User Defined FunctionsShiloh: User Defined Functions

Multi-statement T-SQL routinesMulti-statement T-SQL routines Scalar-valued: Scalar-valued:

Select f(c1) … Select f(c1) … Select … where f2(c2)Select … where f2(c2) In expressionsIn expressions

Table-valued (also called relational): Table-valued (also called relational): Select c1 from f(arg)…Select c1 from f(arg)…

Strongly typed input arguments with return value Strongly typed input arguments with return value No output parametersNo output parameters

Inline relational functionsInline relational functions Effectively a parameterized viewEffectively a parameterized view

Page 11: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Scalar UDFs: ExampleScalar UDFs: Example

CREATE FUNCTION ExtractNamePart(@InName varchar(100), CREATE FUNCTION ExtractNamePart(@InName varchar(100), @part Varchar(10))@part Varchar(10))

RETURNS varchar(30)RETURNS varchar(30)

BEGINBEGIN

DECLARE @offset tinyint ,@return varchar(30)DECLARE @offset tinyint ,@return varchar(30)

SET @offset = charindex(' ', @InName)SET @offset = charindex(' ', @InName)

select @return= case @partselect @return= case @part

WHEN 'First' then substring(@InName, 1, @offset-1)WHEN 'First' then substring(@InName, 1, @offset-1)

WHEN 'Last' then substring(@InName, @offset+1, WHEN 'Last' then substring(@InName, @offset+1, len(@InName))len(@InName))

ELSE NULLELSE NULL

ENDEND

return @returnreturn @return

ENDEND

select dbo.ExtractNamePart('Fadi Fakhouri','Last')select dbo.ExtractNamePart('Fadi Fakhouri','Last')

Page 12: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Column Level CollationsColumn Level Collations

Multi-lingual applications, application Multi-lingual applications, application hosting, server consolidationhosting, server consolidation

Per-database collationsPer-database collations Multiple apps with different collationsMultiple apps with different collations

Per-column collationsPer-column collations Deeper multi-lingual applicationsDeeper multi-lingual applications

Attach, restore databases with different Attach, restore databases with different collations from servercollations from server

Full backwards compatibilityFull backwards compatibility

Page 13: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Column Level Collations: SyntaxColumn Level Collations: Syntax

-- Database-level collation-- Database-level collation

CREATE DATABASE MyDBCREATE DATABASE MyDB

COLLATE Latin1_General_CS_AICOLLATE Latin1_General_CS_AI

-- Column-level collation-- Column-level collation

CREATE TABLE Products (CREATE TABLE Products (

ProductIdProductId char(20), -- db char(20), -- db collationcollation

ProductName char(20) ProductName char(20)

COLLATE French_CS_AS,COLLATE French_CS_AS,

Description char(20) Description char(20)

COLLATE French_CI_AI)COLLATE French_CI_AI)

Page 14: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Column Level Collations: ExamplesColumn Level Collations: Examples

SELECT *SELECT *

FROM PRODUCTSFROM PRODUCTS

ORDER BY ProductName, DescriptionORDER BY ProductName, Description

SELECT *SELECT *

FROM PRODUCTSFROM PRODUCTS

ORDER BY ProductName COLLATE French_CI_AIORDER BY ProductName COLLATE French_CI_AI

SELECT SELECT

DISTINCT ProductName COLLATE French_CI_AIDISTINCT ProductName COLLATE French_CI_AI

FROM ProductsFROM Products

Page 15: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Instead Of TriggersInstead Of Triggers

ApplicationApplication User-defined view updatabilityUser-defined view updatability Before triggersBefore triggers

Trigger executed INSTEAD of Insert, Trigger executed INSTEAD of Insert, Delete, Update actionDelete, Update action

Trigger can be created on view or tableTrigger can be created on view or table Inserted/Deleted tables still availableInserted/Deleted tables still available Statement LevelStatement Level

Page 16: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Instead Of Trigger: ExampleInstead Of Trigger: Example

CREATE TRIGGER AuIns on AuthorsView INSTEAD OF CREATE TRIGGER AuIns on AuthorsView INSTEAD OF INSERTINSERT

ASAS

BEGINBEGIN

INSERT INSERT authorsauthors(au_id,au_fname,au_lname,…)(au_id,au_fname,au_lname,…)

SELECT au_id, SELECT au_id,

dbo.ExtractNamePart(au_name, dbo.ExtractNamePart(au_name, 'First'), 'First'),

dbo.ExtractNamePart(au_name, dbo.ExtractNamePart(au_name, 'last'), 'last'), phone, address, city, phone, address, city, state, zip, state, zip,

contractcontract

FROM FROM insertedinserted

ENDEND

INSERT AuthorsView VALUES(‘John Doe’, …)INSERT AuthorsView VALUES(‘John Doe’, …)

Page 17: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

New DatatypesNew Datatypes

Usage – Columns, Parameters, Usage – Columns, Parameters, Variables, Return typeVariables, Return type

Table TypeTable Type Table valued UDFsTable valued UDFs

BigIntBigInt 8 byte integer8 byte integer

SQL_VariantSQL_Variant Can store any base type (except LOB)Can store any base type (except LOB) Useful for open schema, annotationsUseful for open schema, annotations

Page 18: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Full Text EnhancementsFull Text Enhancements

Change TrackingChange Tracking Maintain list of changes to indexed dataMaintain list of changes to indexed data

Image FilteringImage Filtering Support for MS Office documents, HTMLSupport for MS Office documents, HTML Custom filters (3rd party)Custom filters (3rd party)

Failover ClusteringFailover Clustering Column Level Linguistic analysisColumn Level Linguistic analysis

Columns can have associated languageColumns can have associated language Top-N-By-RankTop-N-By-Rank

Containstable, FreetexttableContainstable, Freetexttable Scalability ImprovementsScalability Improvements

Page 19: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Extended PropertiesExtended Properties

Associate user-defined metadata with SQL objectsAssociate user-defined metadata with SQL objects Richer UI, client-side validationRicher UI, client-side validation Objects Objects (property name, value) (property name, value) Sp_setextendedproperty, Sp_setextendedproperty, Sp_drop extendedproperty, Fn_listextendedpropertySp_drop extendedproperty, Fn_listextendedproperty Example :Example :

exec sp_addextendedproperty 'caption', 'Employee Id', 'user', exec sp_addextendedproperty 'caption', 'Employee Id', 'user', dbo, 'view', foo_view, 'column', iddbo, 'view', foo_view, 'column', id

select * from ::fn_listextendedproperty ('caption', 'user', 'dbo', select * from ::fn_listextendedproperty ('caption', 'user', 'dbo', 'view', 'foo_view', 'column', NULL)'view', 'foo_view', 'column', NULL)

Properties retained on a renameProperties retained on a rename

Page 20: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Other Developer FeaturesOther Developer Features

CheckSumCheckSum Intrinsic for computing a checksum over a set Intrinsic for computing a checksum over a set

of columnsof columns Session contextSession context

Middle-tier apps can set/get user supplied contextMiddle-tier apps can set/get user supplied context Tools Tools

Profiler (larger workloads, performance analysis, Profiler (larger workloads, performance analysis, OLAP events)OLAP events)

Query AnalyzerQuery Analyzer Index Tuning Wizard (Indexed Views)Index Tuning Wizard (Indexed Views)

Page 21: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple instances/machineMultiple instances/machine SecuritySecurity

Scalability, performanceScalability, performance AvailabilityAvailability

Page 22: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Relational To XMLRelational To XML SELECT … FOR XMLSELECT … FOR XML

FOR XML RAW (return an XML rowset)FOR XML RAW (return an XML rowset) FOR XML AUTO (exploit RI, name matching, etc.)FOR XML AUTO (exploit RI, name matching, etc.) FOR XML EXPLICIT (maximal control)FOR XML EXPLICIT (maximal control)

Annotated SchemaAnnotated Schema Maps between XML and relational schema expressed in XMLMaps between XML and relational schema expressed in XML

TemplatesTemplates Encapsulated parameterized queryEncapsulated parameterized query XSL/T supportXSL/T support XPATH supportXPATH support

Direct from DOMDirect from DOM MSXML component hosted in SQL Server kernelMSXML component hosted in SQL Server kernel

Direct URL access (SQL “owned” virtual root)Direct URL access (SQL “owned” virtual root) SELECT … FOR XMLSELECT … FOR XML Annotated schemaAnnotated schema TemplatesTemplates

Page 23: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

XML To RelationalXML To Relational

Bulk load using XMLBulk load using XML Updategram (web delivery Updategram (web delivery

post Shiloh)post Shiloh) Templates and Annotated SchemaTemplates and Annotated Schema DOMDOM

Insert document into SQL Server DOMInsert document into SQL Server DOM Select from DOM rowset and insert Select from DOM rowset and insert

directly into SQL tablesdirectly into SQL tables

Page 24: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

XML: ExampleXML: Example

http://SRV1/nwind?http://SRV1/nwind?sql=SELECT+'<ROOT>';SELECT+DISTINCT+ContactTitsql=SELECT+'<ROOT>';SELECT+DISTINCT+ContactTitle+FROM+Customers+WHERE+ContactTitle+LIKE+'Sale+FROM+Customers+WHERE+ContactTitle+LIKE+'Sa%25'+ORDER+bY+ContactTitle+FOR+XML+AUTO;SELECT%25'+ORDER+bY+ContactTitle+FOR+XML+AUTO;SELECT+'</ROOT>+'</ROOT>''

Result setResult set::<ROOT><ROOT> <Customers ContactTitle="Sales Agent" /> <Customers ContactTitle="Sales Agent" /> <Customers ContactTitle="Sales Associate" /> <Customers ContactTitle="Sales Associate" /> <Customers ContactTitle="Sales Manager" /> <Customers ContactTitle="Sales Manager" /> <Customers ContactTitle="Sales Representative" <Customers ContactTitle="Sales Representative"

/> /> </ROOT></ROOT>

Page 25: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple instances/machineMultiple instances/machine SecuritySecurity

Scalability, performanceScalability, performance AvailabilityAvailability

Page 26: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Multi-InstanceMulti-Instance

Support multiple SQL Servers Support multiple SQL Servers instances on single machineinstances on single machine

Applications:Applications: Application hosting Application hosting Server consolidationServer consolidation SQL Runtime (MSDE)SQL Runtime (MSDE) Used internally in implementation of Used internally in implementation of

shared disk Failovershared disk Failover

Page 27: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Multi-InstanceMulti-Instance

One default Instance (typically server name)One default Instance (typically server name) Multiple named instances (server_name\instance)Multiple named instances (server_name\instance) Named instances are all version 8.0 or aboveNamed instances are all version 8.0 or above 7.0 or 8.0 supported as default database7.0 or 8.0 supported as default database

Shiloh "Named" InstanceShiloh "Named" InstanceShiloh "Named" InstanceShiloh "Named" Instance

Shiloh "Named" InstanceShiloh "Named" Instance

SQL Server 6.5SQL Server 6.5VerswitchVerswitch

Sphinx or Shiloh "Default" InstanceSphinx or Shiloh "Default" Instance

Page 28: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple instances/machineMultiple instances/machine SecuritySecurity

Scalability, performanceScalability, performance AvailabilityAvailability

Page 29: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SecuritySecurity

Initial Setup: Secure Initial Setup: Secure AuditingAuditing EncryptionEncryption

Logins (SSL)Logins (SSL) Internal Server (Crypto APIs)Internal Server (Crypto APIs)

WindowsWindows®® 2000 Kerberos support, 2000 Kerberos support, delegation (linked servers)delegation (linked servers)

Enhancements to Server rolesEnhancements to Server roles C2 Security EvaluationC2 Security Evaluation

Page 30: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple Instances/MachineMultiple Instances/Machine SecuritySecurity

Scalability, PerformanceScalability, Performance AvailabilityAvailability

Page 31: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Scalability: DesktopScalability: Desktop

Multiple Instances of server per systemMultiple Instances of server per system Database level collations Database level collations Microsoft installer based setupMicrosoft installer based setup Reduced start-up time, working setReduced start-up time, working set Simplified licensingSimplified licensing Access integrationAccess integration

Page 32: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Partitioned ViewsPartitioned Views

A partitioned view is a UNION ALL view A partitioned view is a UNION ALL view over two or more tables which have over two or more tables which have partitioning columns with disjoint partitioning columns with disjoint intervalsintervals

Example:Example:

CREATE TABLE CUSTOMER_1K (…, C_W_ID INTCREATE TABLE CUSTOMER_1K (…, C_W_ID INT NOT NULL NOT NULL CHECK (C_W_ID BETWEEN 1 AND 1000) CHECK (C_W_ID BETWEEN 1 AND 1000), , PRIMARY KEYPRIMARY KEY (C_W_ID, …) (C_W_ID, …)

……CREATE TABLE CUSTOMER_10K (…, C_W_ID INTCREATE TABLE CUSTOMER_10K (…, C_W_ID INT NOT NULL NOT NULL CHECK (C_W_ID BETWEEN 9001 AND 10000) CHECK (C_W_ID BETWEEN 9001 AND 10000), , PRIMARY KEYPRIMARY KEY (C_W_ID, …) (C_W_ID, …)

CREATE VIEW CUSTOMER ASCREATE VIEW CUSTOMER AS

SELECT …, C_W_ID FROM CUSTOMER_SELECT …, C_W_ID FROM CUSTOMER_1K1K UNION ALLUNION ALL … …

SELECT …, C_W_ID FROM CUSTOMER_SELECT …, C_W_ID FROM CUSTOMER_1OK1OK

Page 33: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Distributed Partitioned ViewsDistributed Partitioned Views

CREATE V IEW CUSTO M ER AS SELECT <local> ...custom er1K UNIO N ALL ... SELECT nodeNK...custom erNK

node1K node2k node4k node5k nodeNk

Netw ork Sw itchFast Interconnect

...

CREATE V IEW CUSTO M ER AS SELECT node1K...custom er1K UNIO N ALL ... SELECT <local>...custom erNK

Each node in the ‘cluster’ has a single commonEach node in the ‘cluster’ has a single common logical ‘view’ of the tables across all other nodeslogical ‘view’ of the tables across all other nodes

Page 34: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Distributed Partitioned ViewsDistributed Partitioned Views

1.1. Each table in the application is split across the ‘cluster’Each table in the application is split across the ‘cluster’

2.2. Each table is partitioned into disjoint intervals using a Each table is partitioned into disjoint intervals using a CHECK constraintCHECK constraint

3.3. All nodes are linked using OLEDB linked serversAll nodes are linked using OLEDB linked serverssp_addlinkedserver node1sp_addlinkedserver node1sp_setnetname node1, ‘MACHINE NAME’ …sp_setnetname node1, ‘MACHINE NAME’ …

4.4. Views are created for each ‘table’ on each node to link Views are created for each ‘table’ on each node to link together the partitions stored across all nodestogether the partitions stored across all nodes

Page 35: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

PerformancePerformance

Index EnhancementsIndex Enhancements Indexed ViewsIndexed Views Other Index ImprovementsOther Index Improvements

ParallelismParallelism Query Optimizer enhancementsQuery Optimizer enhancements Distributed QueriesDistributed Queries Large memory supportLarge memory support Lots of other enhancementsLots of other enhancements

Page 36: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Indexed ViewsIndexed Views

Decision support workloadsDecision support workloads Content of view is persisted, maintainedContent of view is persisted, maintained View may contain joins, aggregations View may contain joins, aggregations

(some restrictions)(some restrictions) Allows storage of partial resultsAllows storage of partial results Automatically used by optimizer Automatically used by optimizer Example : Example :

Create View Store_Sales as Select StoreId, SUM(qty) Create View Store_Sales as Select StoreId, SUM(qty) From Sales Group By StoreIdFrom Sales Group By StoreId

Create Clustered Index I1 on Store_SalesCreate Clustered Index I1 on Store_SalesSelect StoreId, Qty from Sales where StoreId = ‘100’Select StoreId, Qty from Sales where StoreId = ‘100’

Index Tuning Wizard SupportIndex Tuning Wizard Support

Page 37: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Index EnhancementsIndex Enhancements

Indexes on Computed Columns Indexes on Computed Columns (PK, Unique constraints)(PK, Unique constraints)

Indexes with Asc, Desc columnsIndexes with Asc, Desc columns

Example : Example : Select a, b, c Select a, b, c

From TFrom T

Where a > 10Where a > 10

Order By a asc, b descOrder By a asc, b desc

RE creates, maintains indexesRE creates, maintains indexes Indexes supported on bit columnsIndexes supported on bit columns

Page 38: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

ParallelismParallelism

Enhancements to query parallelismEnhancements to query parallelism More efficient plansMore efficient plans Bitmap filteringBitmap filtering Multi-threaded access to spoolsMulti-threaded access to spools Reduced parallel sort stallsReduced parallel sort stalls Elapsed time costingElapsed time costing Integrated deadlock detectionIntegrated deadlock detection

Insert/Update/DeleteInsert/Update/Delete Index creationIndex creation DBCCDBCC

Page 39: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Query OptimizerQuery Optimizer

Cardinality estimationCardinality estimation Exploit new statisticsExploit new statistics Improve many computationsImprove many computations Handle extreme skewHandle extreme skew

More granular statisticsMore granular statistics Dynamic bucket boundariesDynamic bucket boundaries

Star JoinsStar Joins Snowflake SchemaSnowflake Schema More ‘trivial’ plan casesMore ‘trivial’ plan cases

Page 40: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Distributed QueriesDistributed Queries

New providers – Active DirectoryNew providers – Active Directory™™, , Exchange 2000Exchange 2000

OLEDB Interfaces (Statistics, Constraints)OLEDB Interfaces (Statistics, Constraints) Cost based optimizationCost based optimization

Obtains statistics from remote sourceObtains statistics from remote source Smarter remoting logicSmarter remoting logic

More SQL dialectsMore SQL dialects Minimal SQLMinimal SQL More granular capabilitiesMore granular capabilities

Heterogeneous partitioning viewsHeterogeneous partitioning views

Page 41: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Large Memory SupportLarge Memory Support

Scale-up memory limited applicationsScale-up memory limited applications Up to 64GB on Intel x86 CPUs (PAE)Up to 64GB on Intel x86 CPUs (PAE) Uses Windows 2000 AWE SupportUses Windows 2000 AWE Support Single Buffer CacheSingle Buffer Cache Physical/Virtual Re-Mapping of BuffersPhysical/Virtual Re-Mapping of Buffers sp_configure optionsp_configure option

Page 42: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

SQL Server 2000: SQL Server 2000: Relational Engine Relational Engine (Customer Feedback)(Customer Feedback)

ProgrammabilityProgrammability XML supportXML support ManageabilityManageability

Multiple Instances/MachineMultiple Instances/Machine SecuritySecurity

Scalability, PerformanceScalability, Performance AvailabilityAvailability

Page 43: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

AvailabilityAvailability

Built on top of Built on top of MicrosoftMicrosoft®® Cluster Server Cluster Server Multi-Instance (no virtual layer)Multi-Instance (no virtual layer)

Fixed known issues with 7.0Fixed known issues with 7.0 Installation (Setup)Installation (Setup) Perfmon countersPerfmon counters ToolsTools Licensing informationLicensing information BugsBugs

Page 44: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

AvailabilityAvailability

EnhancementsEnhancements Multiple IP addressesMultiple IP addresses Rolling upgradesRolling upgrades Full-TextFull-Text Disaster RecoveryDisaster Recovery Windows 2000 Data Center supportWindows 2000 Data Center support All components are cluster awareAll components are cluster aware EASY to add/remove nodesEASY to add/remove nodes

Page 45: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

AvailabilityAvailability

FDDI RingFDDI Ring

ServerServer

ServerServer ServerServer

ServerServer

SQL Server ClientSQL Server Client

Server1Server1

Server2Server2

Server3Server3

Server4Server4

Local BinariesLocal Binaries

Local BinariesLocal Binaries

Local BinariesLocal Binaries

Local BinariesLocal Binaries

SQLCLUSTA\INST1SQLCLUSTA\INST1131.107.2.11131.107.2.11

SQLCLUSTD\INST4SQLCLUSTD\INST4131.107.2.14131.107.2.14

SQLCLUSTB\INST2SQLCLUSTB\INST2131.107.2.12131.107.2.12 SQLCLUSTC\INST3SQLCLUSTC\INST3

131.107.2.13131.107.2.13

Clients only see theClients only see thebolded names as thebolded names as the

connection pointsconnection points

DataDataInst1Inst1

DataDataInst2Inst2

DataDataInst4Inst4

DataDataInst3Inst3

DataData

DataData

DataData DataData

Page 46: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

Business OperationsBusiness Operations4-Node Failover4-Node FailoverLog ShippingLog ShippingHet. Partitioning ViewsHet. Partitioning ViewsUser-Defined FunctionsUser-Defined FunctionsXMLXML

Data WarehousingData WarehousingDecision TreesDecision Trees

Large DimensionsLarge DimensionsWeb Hit AnalysisWeb Hit Analysis

English Query for OLAPEnglish Query for OLAP Indexed ViewsIndexed Views

Server-less BackupServer-less Backup

Knowledge ManagementKnowledge ManagementContent FiltersContent Filters

Exchange 2000 IntegrationExchange 2000 Integration

CoreCoreReliabilityReliabilityDelegationDelegationEncryptionEncryption

Schema ReplicationSchema ReplicationMulti-InstanceMulti-Instance

SS2K CESS2K CE

SQL Server 2000SQL Server 2000

Page 47: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206

??

Questions/FeedbackQuestions/Feedback

Page 48: Relational Engine Enhancements James Hamilton Language Processing & Exec Dev Manager SQL Server Microsoft Corp JamesRH@microsoft.com5-206