Useful Sql Stuff wot I learned

Useful Sql Stuff wot I learnedChris HayRoskakori Limitedhttp://silverlightuk.blogspot.com

About Me.NET DeveloperNot a SQL DeveloperWorked on loads of large data projectsCo-Run NxtGenUG Cambridge

http://silverlightuk.blogspot.comwww.nxtgenug.net

2AgendaKnow your DataHeapsPages + Page CachingDBCCSTATISTICSExecution PlansIndexesClusteredNon ClusteredSelectivityStored Procs + Cached Query PlansDynamic SortingSQL ProfilerDynamic FilteringHierarchiesCaching

3Know your DataAlways use a representative data setA query against 10 rows of data will be fastSame query against 1,000,000 rows may not be fastDont need real data, use a data generation scriptOr use Red Gates SQL Data Loader ToolPick correct data size (tinyint, smallint, int, bigint)7MB difference between bigint & int for 1,000,000 rows4No Indexes (Heap)A Table with no IndexesData is stored unorderedAll queries are slowPerforms a Table ScanReads every row in the table Alert Avoid at all costs

Explain High SelectivityExplain Low SelectivityExplain the term table scan5Demo (Heap Queries)CachingClearing CacheExecution PlanStatisticsDrop IndexesExplain Table + Database

Run QueryShow UncachedShow CachedClear Cache + Run Again

Display Estimated Execution PlanInclude Actual Execution PlanShow Table Scan

Show StatisticsExplain Pages, Logical Reads, Physical Reads, Read Ahead Reads

6PagesDifferent Types of Pages (Data, Indexes etc)Data is held in a Data PageA Page is 8Kb8Kb * 8871 = 70968 (69.3Mb)SQL Server reads in pagesCant load just 1 row must load the page

Right Click on the Customers Tablw7Cached PagesEverytime a page is read from disk it is cachedFuture reads are quicker as there is no disk costLarger the Memory = Larger the CacheWith a table scan all pages are cachedBad Table Scan fills up nice cache with stuff we dont wantBad Table Scan, Bad Bad Table ScanClear Cached PagesDBCC DROPCLEANBUFFERSLovely in Dev EnvironmentsCould make you unpopular in production environmentsReduce Cached PagesSplit out infrequently accessed dataEspecially large data e.g varchar(max)

StatisticsIO StatisticsSET STATISTICS IO ONSET STATISTICS IO OFFTime StatisticsSET STATISTICS TIME ONSET STATISTICS TIME OFF

StatisticsHigh CPU Time is BadHigh Logical Reads is BadPhysical Reads are BadRead Ahead Reads are Bad

No Indexes (Heap)A Table with no IndexesData is stored unorderedAll queries are slowHigh and Low Selectivity irrelevantData position in table irrelevantPerforms a Table ScanReads every row in the table

Alert Avoid at all costs

Explain High SelectivityExplain Low SelectivityExplain the term table scan13Run away from Heaps + Table Scans

Clustered Index

Light 1Clustered IndexMy Favourite Index Once found item in Index you have dataGenerally use where possibleSuperfast for range searchesLow selectivity

Non Clustered Index

Non Clustered IndexOnce found item in Index, needs additional lookup to get dataSuperfast for highly selective searchesComparable to clustered indexSlow for range searches (low selectivity)Due to lookupDoesnt need lookup if only columns in indexJust as faster as the clustered indexSelectivityHigh SelectivityA query which returns a small number of rows

Low SelectivityA query which returns a large number of rows

Calculate SelectivityDBCC SHOW_STATISTICS (N'dbo.Customers', IX_Customers)

Stored Procs - prefixingDO - exec dbo.MyStoredProcNameDONT - exec MyStoredProcName

If not suppliedLooks up users default schemaLooks in dbo.schema

Slight Performance GainDo for all new development

Stored Procs Cached Query PlansQuery plan determines type of Joins / Indexes etcSQL Server calculates best plan based on statisticsNew Statement = New Query PlanCached at a statement levelQuery Plans are reusableStored Procs - DBCC FREEPROCCACHEClears all Cached Query Plans in the DatabaseShould use when Performance TuningDangerous to use in Production

Stored Procs With RecompileStored Procedure Level RecompilationGood for multi statement stored procsMark up Stored proc WITH RECOMPILE

23Stored Procs With Recompile

24Stored Procs - Option RecompileStatement Level Recompilation OnlyGood for multi statement stored procs

Stored Procs - OPTIMIZEStatement LevelOptimizes Query Plan for parameterIgnores actual passed parameter

Dynamic SortingAvoiding sorting if possible

Sometimes need the ability to change sort column / orderE.g. Grid in a Web Application27Dynamic Sorting Bad Example

Alert Avoid if possibleDynamic SortingDEMO Optimal PlansDEMO Case Stored ProcDynamic Sorting Case is BadGenerates a non optimal query plan

Dynamic Sorting - SolutionsLots of Static Stored ProcsListCustomersByCustomerIdListCustomersByMobileQuery Plan CachedUnwieldy + High Maintenance Cost

31Dynamic Sorting Dynamic SQLDynamic SQLQuery Plan CachedSecurity / SQL Injection Concerns OptionsManual (be very careful, SQL Injection)LINQ2SQL (SQL Injection protected)ADO Entity Framework (SQL Injection protected)32Dynamic Sorting Dynamic SQLDEMO Dynamic SQLDEMO LINQ2SQLSQL ProfilerShould always run whilst developingView query with parametersView actual statisticsCareful running in productionDEMO SQL ProfilerApplication NameCan filter your connections in profilerUseful with shared dev databasesDEMO Application Name

Dynamic FilteringUsual Solution (Very Bad)

Same solutions as Dynamic SortingLots of static stored proceduresDynamic SQLLINQ 2 SQLADO .NET Entity Framework

36HierarchiesParent / Child RelationshipsManagers / EmployeesCategories / Sub Categories

Hierarchies

38HierarchiesSQL 2000 Very messy code to doSQL 2005 / 2008 Very easyRecursive Common Table Expression (CTE)

Hierarchies

40CachingDatabase is always application bottleneckCache Data wherever possible

Demo: Caching (VoxPeeps)

alert: Do when ever possible

41Wot we coveredKnow your dataHeapsPages + Page CachingDBCCSTATISTICSExecution PlansIndexesClusteredNon ClusteredSelectivityStored Procs + Cached Query PlansDynamic SortingSQL ProfilerDynamic FilteringHierarchiesCaching

Explain High SelectivityExplain Low SelectivityExplain the term table scan42Useful Sql Stuff wot I learnedChris HayRoskakori Limitedhttp://silverlightuk.blogspot.com