nw_sql server internals & architecture

SQL Server Internals & Architecture

Naomi Williams, SQL DBATwitter @naomithesqldbaLinkedIn www.linkedin.com/in/nfwilliams/

Why is there ACID in my SQL Server?

ACID properties of Transactions

AtomicConsistent IsolatedDurable

3

Welcome to SQL Server! Where does the path begin?

SELECT SQL Server Interface aka API

TDS packets

SNI (SQL Server Client Network Interface)

SNI (SQL Server Server Network Interface)

Protocols

TCP/IP

Shared Memory

Named Pipes

Virtual Interface Adapter (VIA)

SQLOS

5

SQLOS, Workers and Schedulers

SQLOS creates a set of schedulers upon start equal to the number of logical CPU’s.

Each scheduler manages workers, for every task to execute it’s assigned a worker that is in an idle state.

Workers do not move between schedulers, tasks are never moved between workers. However SQLOS can create child tasks and assign them different workers.

Tasks can have one of six states

Pending; task is waiting for an available worker

Done; task is completed

Running; task is currently executing

Runnable; task is waiting for the scheduler

Suspended; task is waiting for external event or resource

Spinloop; task is processing a spinlock

SQLOS, Workers and Schedulers, cont

6

Running state

Suspended queue and suspended state

Runnable queue and runnable state

SELECT SQL Server Interface aka API

TDS packets



Protocols

TCP/IP

Shared Memory

Named Pipes


Database EngineCmd Parser Optimizer

Query PlanQuery Tree

Language Event

Query Executor

Buffer Pool

Data Cache Language

Hash of execution plan

Does the hash exist in the Plan Cache?

If query hash does not exist in Plan Cache

YES! Query has plan exists!

Plan Cache

SQLOS

Project (*)

Filter (o.date=‘2014-12-12’)

Inner Join c.cid=o.cid

Get (customers) as C

Get (orders) as O

Logical Query TreeSelect * from Customers c inner join Orders o on c.cid=o.cid where o.date=‘2014-12-12’;

SELECTSQL Server Interface aka API

TDS packets



Protocols

TCP/IP

Shared Memory

Named Pipes




Language Event

Query Executor

Buffer Pool

Data Cache

Plan Cache

SQLOS

10

Query Optimizer

Parse and logical query treeDoes it make sense and how can I get there?

Binding, do they exist?Do the objects exist? Create path in binary from algebrizer for optimizer.

Generate physical query planMost efficient plan (path) to the data.

Physical Query Tree

SELECTSQL Server Interface aka API

TDS packetsSNI (SQL Server

Client Network Interface)


Protocols

TCP/IP

Shared Memory

Named Pipes




Language Event

Query Executor

Data File

Transaction Log File Storage

Engine

Buffer Manager

Access MethodsTransaction Manager

Buffer Pool

Data Cache

OLE DBDoes Data exist in Cache?

Retrieves data from disk to cache

Plan Cache

SQLOS

Insert, Update, DeleteSQL Server Interface aka API




Protocols

TCP/IP

Shared Memory

Named Pipes




Language Event

Query Executor

Data File


Engine

Buffer Manager


Lock and Log manager

Buffer Pool

Data Cache

OLE DB

Plan Cache

SQLOS

CheckPoint

WAIT, WAIT WAIT

Data File

SOS_Scheduler_YieldAsync_Network

_IO

Writelog, LogBuffer

PageIOLatch_x, Async_IO_Complet

ion, IO_Completion

Pagelatch_x, Latch_x,

Resource_Semaphore

LocksLCK_x, LCK_M_x

Latches

OLEDB

BackupIO CXPacket

IO_Completion

MSQL_XPMSQL_DQ

PreEmptive_OS_

ThreadPool

WAIT, WAIT WAITSQL Server Interface aka API




Protocols

TCP/IP

Shared Memory

Named Pipes




Language Event

Query Executor

Data File


Engine

Buffer Manager



Buffer Pool

Data Cache

OLE DB

Plan Cache

SOS_Scheduler_Yield, CXPacket

Async_Network_IO

SQLOS

Writelog, LogBuffer

PageIOLatch_x, Async_IO_Complet

ion, IO_Completion

Pagelatch_x, Latch_x,

Resource_Semaphore

Locks

LCK_x, LCK_M_x

Latches

16

Data Cache and LRU-K

A B C A B C

LAZY WRITER

Data FileA B CBuffer Pool

Data Cache Lazy Writer

0

0

SQLOSFree Buffer List Every buffer has a

header Last two times

the page was referenced

Status information

17

Plan Cache and LRU

A B C A B C9 5 2

Buffer Pool

Plan Cache

Resource Monitor

0

0

SQLOS

A B C7 3 0 9 2

18

SQL Server 2012 Indexes that include columns of large object (LOB) data—

varchar(max), nvarchar(max), varbinary(max), or XML data—can now be built, rebuilt, or dropped online.

Adding table columns that contain default values is now a metadata-only operation

Columnstore Indexes mainly for increased performance in data warehouses

The new spatial features in SQL Server 2012 represent a significant milestone in the evolution of spatial data support in SQL Server.

The FileTable feature builds on top of the SQL Server FILESTREAM technology to bring support for the Windows file namespace and compatibility with Windows applications to the file data stored in SQL Server.

19

SQL Server 2014 In SQL Server 2014, statistics are not created by scanning all rows in the table when a partitioned

index is created or rebuilt.

The new cardinality estimator incorporates assumptions and algorithms that work well on modern OLTP and data warehousing workloads.

The individual partitions of partitioned tables can now be rebuilt. The WAIT_AT_LOW_PRIORITY option also allows you to configure the termination of blocking

processes related to that rebuild statement

New features are available for columnstore indexes The Buffer Pool Extension

Incremental Statistics especially great for partitioned indexes

Resource Governor Enhancements for Physical IO Control

Hekaton

Delayed Durability is not reducing the workload at all

http://msdn.microsoft.com/en-us/library/dn133176.aspx

HekatonSQL Server Interface aka API




Protocols

TCP/IP

Shared Memory

Named Pipes




Language Event

Query Executor

Data File


Engine

Buffer Manager



Buffer Pool

Data Cache

OLE DB

Plan Cache

SQLOS

CheckPoint

Natively compiled stored procedures and

schema

In-Memory OLTP Compiler

Memory Optimized Tables and Indexes

21

SummaryACIDSQLOSComponents of database

engineComponents of storage

engineCache agingLimitations of Hekaton

nw_sql server internals & architecture

Documents