nw_sql server internals & architecture
Post on 16-Aug-2015
38 Views
Preview:
TRANSCRIPT
SQL Server Internals & Architecture
Naomi Williams, SQL DBATwitter @naomithesqldbaLinkedIn www.linkedin.com/in/nfwilliams/
Why is there ACID in my SQL Server?
ACID properties of Transactions
AtomicConsistent IsolatedDurable
3
Welcome to SQL Server! Where does the path begin?
SELECT SQL Server Interface aka API
TDS packets
SNI (SQL Server Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
SQLOS
5
SQLOS, Workers and Schedulers
SQLOS creates a set of schedulers upon start equal to the number of logical CPU’s.
Each scheduler manages workers, for every task to execute it’s assigned a worker that is in an idle state.
Workers do not move between schedulers, tasks are never moved between workers. However SQLOS can create child tasks and assign them different workers.
Tasks can have one of six states
Pending; task is waiting for an available worker
Done; task is completed
Running; task is currently executing
Runnable; task is waiting for the scheduler
Suspended; task is waiting for external event or resource
Spinloop; task is processing a spinlock
SQLOS, Workers and Schedulers, cont
6
Running state
Suspended queue and suspended state
Runnable queue and runnable state
SELECT SQL Server Interface aka API
TDS packets
SNI (SQL Server Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Buffer Pool
Data Cache Language
Hash of execution plan
Does the hash exist in the Plan Cache?
If query hash does not exist in Plan Cache
YES! Query has plan exists!
Plan Cache
SQLOS
Project (*)
Filter (o.date=‘2014-12-12’)
Inner Join c.cid=o.cid
Get (customers) as C
Get (orders) as O
Logical Query TreeSelect * from Customers c inner join Orders o on c.cid=o.cid where o.date=‘2014-12-12’;
SELECTSQL Server Interface aka API
TDS packets
SNI (SQL Server Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Buffer Pool
Data Cache
Plan Cache
SQLOS
10
Query Optimizer
Parse and logical query treeDoes it make sense and how can I get there?
Binding, do they exist?Do the objects exist? Create path in binary from algebrizer for optimizer.
Generate physical query planMost efficient plan (path) to the data.
Physical Query Tree
SELECTSQL Server Interface aka API
TDS packetsSNI (SQL Server
Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Data File
Transaction Log File Storage
Engine
Buffer Manager
Access MethodsTransaction Manager
Buffer Pool
Data Cache
OLE DBDoes Data exist in Cache?
Retrieves data from disk to cache
Plan Cache
SQLOS
Insert, Update, DeleteSQL Server Interface aka API
TDS packetsSNI (SQL Server
Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Data File
Transaction Log File Storage
Engine
Buffer Manager
Access MethodsTransaction Manager
Lock and Log manager
Buffer Pool
Data Cache
OLE DB
Plan Cache
SQLOS
CheckPoint
WAIT, WAIT WAIT
Data File
SOS_Scheduler_YieldAsync_Network
_IO
Writelog, LogBuffer
PageIOLatch_x, Async_IO_Complet
ion, IO_Completion
Pagelatch_x, Latch_x,
Resource_Semaphore
LocksLCK_x, LCK_M_x
Latches
OLEDB
BackupIO CXPacket
IO_Completion
MSQL_XPMSQL_DQ
PreEmptive_OS_
ThreadPool
WAIT, WAIT WAITSQL Server Interface aka API
TDS packetsSNI (SQL Server
Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Data File
Transaction Log File Storage
Engine
Buffer Manager
Access MethodsTransaction Manager
Lock and Log manager
Buffer Pool
Data Cache
OLE DB
Plan Cache
SOS_Scheduler_Yield, CXPacket
Async_Network_IO
SQLOS
Writelog, LogBuffer
PageIOLatch_x, Async_IO_Complet
ion, IO_Completion
Pagelatch_x, Latch_x,
Resource_Semaphore
Locks
LCK_x, LCK_M_x
Latches
16
Data Cache and LRU-K
A B C A B C
LAZY WRITER
Data FileA B CBuffer Pool
Data Cache Lazy Writer
0
0
SQLOSFree Buffer List Every buffer has a
header Last two times
the page was referenced
Status information
17
Plan Cache and LRU
A B C A B C9 5 2
Buffer Pool
Plan Cache
Resource Monitor
0
0
SQLOS
A B C7 3 0 9 2
18
SQL Server 2012 Indexes that include columns of large object (LOB) data—
varchar(max), nvarchar(max), varbinary(max), or XML data—can now be built, rebuilt, or dropped online.
Adding table columns that contain default values is now a metadata-only operation
Columnstore Indexes mainly for increased performance in data warehouses
The new spatial features in SQL Server 2012 represent a significant milestone in the evolution of spatial data support in SQL Server.
The FileTable feature builds on top of the SQL Server FILESTREAM technology to bring support for the Windows file namespace and compatibility with Windows applications to the file data stored in SQL Server.
19
SQL Server 2014 In SQL Server 2014, statistics are not created by scanning all rows in the table when a partitioned
index is created or rebuilt.
The new cardinality estimator incorporates assumptions and algorithms that work well on modern OLTP and data warehousing workloads.
The individual partitions of partitioned tables can now be rebuilt. The WAIT_AT_LOW_PRIORITY option also allows you to configure the termination of blocking
processes related to that rebuild statement
New features are available for columnstore indexes The Buffer Pool Extension
Incremental Statistics especially great for partitioned indexes
Resource Governor Enhancements for Physical IO Control
Hekaton
Delayed Durability is not reducing the workload at all
HekatonSQL Server Interface aka API
TDS packetsSNI (SQL Server
Client Network Interface)
SNI (SQL Server Server Network Interface)
Protocols
TCP/IP
Shared Memory
Named Pipes
Virtual Interface Adapter (VIA)
Database EngineCmd Parser Optimizer
Query PlanQuery Tree
Language Event
Query Executor
Data File
Transaction Log File Storage
Engine
Buffer Manager
Access MethodsTransaction Manager
Lock and Log manager
Buffer Pool
Data Cache
OLE DB
Plan Cache
SQLOS
CheckPoint
Natively compiled stored procedures and
schema
In-Memory OLTP Compiler
Memory Optimized Tables and Indexes
21
SummaryACIDSQLOSComponents of database
engineComponents of storage
engineCache agingLimitations of Hekaton
top related