sql server 2000 research series - architecture overview

Technical Integration DepartmentTechnical Integration DepartmentSystem Analysis & Training GroupSystem Analysis & Training Group

Jerry YangJerry YangAugust 25, 2005August 25, 2005

SQL Server 2000 Research Series SQL Server 2000 Research Series

Architecture OverviewArchitecture Overview

IntroductionIntroduction Operating Environment Operating Environment The SQL Server EngineThe SQL Server Engine Memory ManagementMemory Management Transaction Logging And Recovery Transaction Logging And Recovery SummarySummary

AgendaAgenda

IntroductionIntroduction Operating EnvironmentOperating Environment The SQL Server EngineThe SQL Server Engine Memory ManagementMemory Management Transaction Logging And RecoveryTransaction Logging And Recovery SummarySummary

Architectural OverviewArchitectural Overview

IntroductionIntroduction What Is It ?What Is It ? Why Analyze Architecture ?Why Analyze Architecture ?

Processing ProceduresProcessing Procedures Performance IssuesPerformance Issues

Hardware DeployHardware Deploy Software ImplementationSoftware Implementation

How Should We Proceed ?How Should We Proceed ? DBMS StructureDBMS Structure SQL Server ComponentsSQL Server Components RelationshipsRelationships

DBMS – Database Management SystemsDBMS – Database Management Systems

IntroductionIntroduction

Application Programs/Queries

Software to ProcessQueries/Programs

Software to AccessStored Data

Users/Programmers

DATABASESYSTEM

DBMSSOFTWARE

Stored DatabaseDefinition

(Meta-Data)

StoredDatabase

Service ManagerService Manager Enterprise ManagerEnterprise Manager Query AnalyzerQuery Analyzer SQL Server ProfilerSQL Server Profiler DTS - Data Transformation ServicesDTS - Data Transformation Services Client Network UtilityClient Network Utility

Operating EnvironmentOperating Environment

Service ManagerService Manager Enterprise ManagerEnterprise Manager Query AnalyzerQuery Analyzer SQL Server ProfilerSQL Server Profiler DTS - Data Transformation ServicesDTS - Data Transformation Services Client Network Utility Client Network Utility


Service ManagerService Manager


Enterprise ManagerEnterprise Manager


Enterprise Manager (Continued)Enterprise Manager (Continued)


Query AnalyzerQuery Analyzer


Query Analyzer (Continued)Query Analyzer (Continued)


The SQL Server EngineThe SQL Server Engine SQL Server ArchitectureSQL Server Architecture

Net-LibraryNet-Library User Mode SchedulerUser Mode Scheduler Relational EngineRelational Engine OLE & Non-OLE DB CommunicationOLE & Non-OLE DB Communication Storage EngineStorage Engine I/O ManagerI/O Manager Win 32 APIWin 32 API

The SQL Server EngineThe SQL Server Engine SQL Server Architecture (Continued)SQL Server Architecture (Continued)

The SQL Server EngineThe SQL Server Engine Net-LibraryNet-Library

FunctionalityFunctionality An abstraction layer An abstraction layer

enables SQL Server to enables SQL Server to read from and write to read from and write to many different network many different network protocols.protocols.

AdvantageAdvantage Support many network Support many network

protocols without having protocols without having to change the core server to change the core server code.code.

The SQL Server EngineThe SQL Server Engine Net-Library (Continued)Net-Library (Continued)

Primary Net-LibrariesPrimary Net-Libraries MembersMembers

• OLE DB Provider for SQL ServerOLE DB Provider for SQL Server• SQL Server ODBC DriverSQL Server ODBC Driver

Super Socket & Shared MemorySuper Socket & Shared Memory• Inter-computer ConnectionInter-computer Connection• Local ConnectionLocal Connection

Secondary Net-LibrariesSecondary Net-Libraries MembersMembers

• TCP/IPTCP/IP• IPX/SPXIPX/SPX• ETC.ETC.

The SQL Server EngineThe SQL Server Engine Net-Library (Continued)Net-Library (Continued)


The SQL Server EngineThe SQL Server Engine Relational EngineRelational Engine

FunctionalityFunctionality Comprises all the Comprises all the

components necessary components necessary to parse and optimize to parse and optimize any query.any query.

Manages the execution Manages the execution of queries as it requests of queries as it requests data from the storage data from the storage engine in terms of OLE engine in terms of OLE DB row sets and then DB row sets and then processes the row sets processes the row sets returned.returned.


ODSODS (Open Data Services)


ODS (Continued)ODS (Continued) FunctionalityFunctionality

• An interface between server Net-Library and server-based An interface between server Net-Library and server-based applications.applications.

• Connection Handler: Connection Handler:

<1><1>. Listens for new connection. Listens for new connection

<2><2>. Cleans up failed connections. Cleans up failed connections

<3><3>. Acknowledges cancellations of commands. Acknowledges cancellations of commands

<4><4>. Returns result sets, messages, status values to the client. Returns result sets, messages, status values to the client


Command ParserCommand Parser


Command Parser (Continued)Command Parser (Continued) FunctionalityFunctionality

• Handles language events raised by ODS. Handles language events raised by ODS. • Command Syntax CheckerCommand Syntax Checker• Error Raiser & IdentifierError Raiser & Identifier• Command Translator – Internal Format (Query Tree)Command Translator – Internal Format (Query Tree)

NoteNote• The only one can access the source of the statement.The only one can access the source of the statement.


OptimizerOptimizer


Optimizer (Continued)Optimizer (Continued) FunctionalityFunctionality

• Takes the Query Tree from the Command Parser and prepared it Takes the Query Tree from the Command Parser and prepared it for execution.for execution.

• Execution Plan GenerationExecution Plan Generation

<1><1>. Normalization. Normalization

<2><2>. Hardware Part: . Hardware Part: Memory, CPU, I/O, ETC.Memory, CPU, I/O, ETC.

<3><3>. Table Part:. Table Part: Table, Column, Index, ETC.Table, Column, Index, ETC.


SQL ManagerSQL Manager


SQL Manager (Continued)SQL Manager (Continued) FunctionalityFunctionality

• Be responsible for everything having to do with managing Be responsible for everything having to do with managing stored procedures and their plans.stored procedures and their plans.

• Recompilation – Based on changes in the object’s schemaRecompilation – Based on changes in the object’s schema• Caching of procedure plan managementCaching of procedure plan management• Auto-parameterized QueryAuto-parameterized Query

SELECT * FROM CustomerSELECT * FROM Customer

WHERE Customer_ID = ‘1234567’WHERE Customer_ID = ‘1234567’

SELECT * FROM CustomerSELECT * FROM Customer

WHERE Customer_ID = @paramWHERE Customer_ID = @param


Expression ManagerExpression Manager


Expression Manager (Continued)Expression Manager (Continued) FunctionalityFunctionality

• Handles computation, comparison, and data movement.Handles computation, comparison, and data movement.

SELECT @var_Salary = Salary * 0.9SELECT @var_Salary = Salary * 0.9

FROM EmployeeFROM Employee

WHERE Employee_No = ‘013456’WHERE Employee_No = ‘013456’


Query ExecutorQuery Executor FunctionalityFunctionality

• Runs Execution PlanRuns Execution Plan• Interacts with Storage EngineInteracts with Storage Engine

<1><1>. Modifies Data. Modifies Data

<2><2>. Retrieves Data. Retrieves Data

<3><3>. Manages Transaction. Manages Transaction

<4><4>. Handles Locking. Handles Locking

The SQL Server EngineThe SQL Server Engine Storage EngineStorage Engine

FunctionalityFunctionality Comprises the components needed to actually access and modify data on disk.Comprises the components needed to actually access and modify data on disk.


Access Methods ManagerAccess Methods Manager


Access Methods Manager (Continued)Access Methods Manager (Continued) FunctionalityFunctionality

• Sets up and requests scans of data pages / index pages Sets up and requests scans of data pages / index pages and prepared the OLE (Object Linking and Embedding) and prepared the OLE (Object Linking and Embedding) DB row sets to return to the Relational Engine.DB row sets to return to the Relational Engine.

• Requester: Access Methods ManagerRequester: Access Methods Manager• Executer: Row Operations Manager & Index ManagerExecuter: Row Operations Manager & Index Manager• Queries, Updates, and DeletesQueries, Updates, and Deletes


Page Manager & Text ManagerPage Manager & Text Manager


Page Manager & Text Manager (Continued)Page Manager & Text Manager (Continued) FunctionalityFunctionality

• The Page Manager and Text Manager cooperate to The Page Manager and Text Manager cooperate to manage a collection of pages as named database.manage a collection of pages as named database.

Type of Disk PagesType of Disk Pages• Data PagesData Pages• Text / Image PagesText / Image Pages• Index PagesIndex Pages• Page Free Space (PFS) PagesPage Free Space (PFS) Pages• Global Allocation Map (GAM) PagesGlobal Allocation Map (GAM) Pages• Index Allocation Map (IAM) PagesIndex Allocation Map (IAM) Pages• Bulk Changed Map PagesBulk Changed Map Pages• Differential Changed Map PagesDifferential Changed Map Pages

For three kinds of datatypes storage:(1).text(2).ntext(3).image

All user data, except for text, ntext, and image datatypes.


Buffer ManagerBuffer Manager


Buffer Manager (Continued)Buffer Manager (Continued) FunctionalityFunctionality

• Handles the in-memory version of each physical disk page and Handles the in-memory version of each physical disk page and provides all other modules access to it.provides all other modules access to it.

NoteNote• When a page is needed for a process, it must exist in memory in When a page is needed for a process, it must exist in memory in

the buffer pool.the buffer pool.• If the page isn’t there, a physicalIf the page isn’t there, a physical I I / / OO is performed to get it. is performed to get it.


Transaction ManagerTransaction Manager


Transaction Manager (Continued)Transaction Manager (Continued) FunctionalityFunctionality

• Guarantees the consistency and recoverability of SQL Server Guarantees the consistency and recoverability of SQL Server databases.databases.

• Coordinates the following items:Coordinates the following items:

<1><1>. Log Manager. Log Manager

<2><2>. Buffer Manager. Buffer Manager

<3><3>. Lock Manager . Lock Manager

<4><4>. Recovery Module. Recovery Module

NoteNote• SUSPENDED!SUSPENDED!

Chapter: Transaction Logging And RecoveryChapter: Transaction Logging And Recovery


Lock ManagerLock Manager


Lock Manager (Continued)Lock Manager (Continued) FunctionalityFunctionality

• Guards data and the internal resources that make it possible for Guards data and the internal resources that make it possible for many users to simultaneously access the database and not be many users to simultaneously access the database and not be severely affected by others’ uses.severely affected by others’ uses.

Two Separate Locking SystemsTwo Separate Locking Systems• Locking System Locking System I – I – Fully Shared Data Table Fully Shared Data Table• Locking System Locking System II – II – Restricted System Data Restricted System Data

The SQL Server EngineThe SQL Server Engine Relational / Storage Engines & DBMS Relational / Storage Engines & DBMS

Memory PoolsMemory Pools What Is Memory Pools ?What Is Memory Pools ? Why Use It ?Why Use It ? Pool TypesPool Types

Buffer PoolBuffer Pool• 4 ~ 10 MB4 ~ 10 MB• 1 Buffer = 1 Page (8 KB)1 Buffer = 1 Page (8 KB)

Operating System MemoryOperating System Memory• MAX – 4 ~ 10 MBMAX – 4 ~ 10 MB• Need Memory Size > 8 KBNeed Memory Size > 8 KB

Memory ManagementMemory Management

Buffer ManagerBuffer Manager In-Memory Pages Access - HashingIn-Memory Pages Access - Hashing

Hash ListsHash Lists Hash BucketsHash Buckets

A Hash Bucket is a structure in memory that A Hash Bucket is a structure in memory that contains an array of pointers to the buffer page.contains an array of pointers to the buffer page.

Hashing FunctionHashing Function Hashing is a technique that uniformly maps a key Hashing is a technique that uniformly maps a key

via a hash function across a set of Hash Buckets.via a hash function across a set of Hash Buckets.


Buffer Sweeper - LazywriterBuffer Sweeper - Lazywriter Buffer Pool ManagementBuffer Pool Management Reference CountReference Count

Data / Index PagesData / Index Pages Expensive PagesExpensive Pages

• Ex: Stored Procedure PlansEx: Stored Procedure Plans

Free ListFree List


Corporations of ManagersCorporations of Managers Transaction ManagerTransaction Manager Buffer ManagerBuffer Manager Log ManagerLog Manager Lock ManagerLock Manager Recovery ModuleRecovery Module

Transaction Logging And Recovery Transaction Logging And Recovery

Transaction LoggingTransaction Logging DefinitionDefinition

Transaction log is a set of files associated with a Transaction log is a set of files associated with a database at the time the database is created or database at the time the database is created or altered.altered.

LSNLSN Log Sequence NumberLog Sequence Number


RecoveryRecovery CheckpointCheckpoint

Database-by-database BasisDatabase-by-database Basis All committed transactions are written to the database.All committed transactions are written to the database. Triggered ConditionsTriggered Conditions

• Database Owner Issues A Ckeckpoint Command Database Owner Issues A Ckeckpoint Command • Log Is Getting Full (more than 70% of capacity)Log Is Getting Full (more than 70% of capacity)• Long Recovery Time Is EstimatedLong Recovery Time Is Estimated


Recovery (Continued)Recovery (Continued) Recovery AlgorithmRecovery Algorithm

Phase 1: AnalysisPhase 1: Analysis Phase 2: Redo (rollforward)Phase 2: Redo (rollforward) Phase 3: Undo (rollback)Phase 3: Undo (rollback)


Recovery (Continued)Recovery (Continued)


SummarySummary What Do You Need to Know Today… What Do You Need to Know Today…

Query Executer

Fundamentals of Database SystemsFundamentals of Database Systems Author:Author: Elmasri / NavatheElmasri / Navathe Publisher:Publisher: Addison-Wesley PublishingAddison-Wesley Publishing

CompanyCompany

Inside of Microsoft SQL Server 2000Inside of Microsoft SQL Server 2000 Author:Author: Kalen DelaneyKalen Delaney Publisher:Publisher: Microsoft PressMicrosoft Press

ReferenceReference

Any Question?Any Question?

sql server 2000 research series - architecture overview

Technology

server netlibrary

serverbased applications

core server code

primary netlibraries

recovery summary agenda

db row sets

stored procedures

software implementation