tut - sql server transaction logs

8/10/2019 TUT - SQL Server Transaction Logs

1/19

Introduction to the SQL Server Transaction Log

Overview

In this tutorial, well go over some of the tasks you can do to manage the transaction log. Thetransaction log is very important to SQL Server and there are books alone that discuss how tomanage the log so this will be an introduction tutorial to the transaction log.

Some questions that I hope to answer in this tutorial include:

What is the transaction log? Why does SQL Server require a transaction log? What is a transaction? What does BEGIN TRAN, ROLLBACK TRAN, and COMMIT TRAN mean? What is a nested transaction?

How does Mirroring, Replication, and Log Shipping use the transaction log? Why does the transaction log grow? What is a Virtual Log File (VLF)? How to monitor transaction log growth. How to shrink the transaction log. How large should the transaction log be? Do multiple transaction logs help performance? What are the recovery models and how does the log use them? How to recover data using a transaction log backup.

What is the transaction log?

Where do I start? The transaction log is an integral part of SQL Server. Every database has atransaction log that is stored within the log file that is separate from the data file. A transactionlog basically records all database modifications. When a user issues an INSERT, for example, it islogged in the transaction log. This enables the database to roll back or restore the transaction ifa failure were to occur and prevents data corruption. For example, lets say Sue is using anapplication and inser ts 2000 rows of data. While SQL Server is processing this data lets saysomeone pulls the plug on the server. (Again, this is just an example, I hope nobody wouldactually do this). Because the INSERT statement was writing to the transaction log and it knowsa failure occurred it will roll back the statement. If this wasnt put in place, could you imagine

having to sift through the data to see how many rows it inserted and then change the code toinsert the remaining rows? Or even better, what if the application inserted random columns inno order and you had to determine what data was inserted and what data was left out? Thiscould take forever!

Log entries are sequential in nature. The transaction log is split up into small chunks calledvirtual log files, which we will discuss in a later section. When a virtual log file is full,transactions automatically move to the next virtual log file. As long as the log records at the


2/19

beginning of the transaction log have been truncated when logging reaches the end of the log,it will circle back around to the start and will overwrite what was there before:

Why does SQL Server require a transaction log?

SQL Server needs a transaction log? No, SQL Server HAS to have a transaction log to work?Without a log file, your database will not work.at all.

In the next section well talk more about what a transaction is, but this section will cover thetransaction log.

The transaction log supports the following:

ROLLBACK TRANSACTION - If a user or application issues the ROLLBACK statement, or if the DBengine detects a failure, the log records are used to roll back the transaction. We will discussBEGIN, COMMIT, and ROLLBACK TRANSACTION later in this tutorial.

Recover Incomplete Transactions - If you have ever started SQL Server from a failure you mayhave noticed databases in the (In Recovery) mode:

This is an indication that SQL Server is rolling back transactions that did not complete beforethe SQL Server was restarted or it is rolling forward all modifications that were recorded in thelog but not written to the data file. You may also see this if you have started a restore WITHRECOVERY.


3/19

Rolling a restored DB, file, filegroup, or page forward to the point of failure If SQL Serverwere to fail and you need to restore the database back to the point in which the failureoccurred you can as long as you are using the FULL recovery model. Start with a full backup,then apply the latest differential, and the subsequent transaction log backups up to the point offailure. We will go into more detail later in this tutorial. You can find more about this here .

High availability solutions - Transactional replication, mirroring, and log shipping all use thetransaction log. We will discuss how the log is used later in this tutorial.

What is a transaction?

A transaction can be defined in many different ways and Ive always had this question come upin interviews. Basically, a transaction is a unit of work that is performed against a database. Thiswork can be performed manually, such as an UPDATE statement you issue in SQL ServerManagement Studio or an application that INSERTS data into the database. These are all

transactions.

SQL Server supports the following transaction modes:

Autocommit transactions - Each individual statement is a transaction.

Explicit transactions - Each transaction is explicitly started with the BEGIN TRANSACTIONstatement and explicitly ended with a COMMIT or ROLLBACK statement.

Implicit transactions A new transaction is implicitly started when the prior transactioncompletes, but each transaction is explicitly completed with a COMMIT or ROLLBACK

statement.

Batch-scoped transactions - Applicable only to multiple active result sets (MARS), a Transact-SQL explicit or implicit transaction that starts under a MARS session becomes a batch-scopedtransaction. A batch-scoped transaction that is not committed or rolled back when a batchcompletes is automatically rolled back by SQL Server.

You may have heard of the ACID properties. These apply to transactions as well:

Atomicity - ensures that all operations within the work unit are completed successfully,otherwise the transaction is aborted at the point of failure and previous operations are rolledback to their former state.

Consistency - ensures that the database properly changes states upon a successfully committedtransaction.

Isolation enables transactions to operate independently of and transparent to each other.
http://www.mssqltips.com/sqlservertutorial/119/sql-server-point-in-time-restore/http://www.mssqltips.com/sqlservertutorial/119/sql-server-point-in-time-restore/http://www.mssqltips.com/sqlservertutorial/119/sql-server-point-in-time-restore/http://www.mssqltips.com/sqlservertutorial/119/sql-server-point-in-time-restore/


4/19

Durability - ensures that the result or effect of a committed transaction persists in case of asystem failure.

Last, but not least, SQL Server supports transaction control. Below is a short description ofeach, but well go over them in more detail in the next section. Note that transaction controls

are only used with DML commands.

BEGIN TRANSACTION - the starting point of a transaction

ROLLBACK TRANSACTION - roll back a transaction either because of a mistake or a failure

COMMIT TRANSACTION - save changes to the database

What does BEGIN TRAN, ROLLBACK TRAN, and COMMIT TRAN mean?When creating a SQL Statement by default, for example, SELECT * FROMHumanResources.Employee, SQL Server will run this statement and immediately return theresults:

If you were to add BEGIN TRANSACTION (or BEGIN TRAN) before the statement it automaticallymakes the transaction explicit and holds a lock on the table until the transaction is eithercommitted or rolled back.

BEGIN TRANSACTION marks the starting point of an explicit, local transaction. - MS

For example, when I issue a DELETE or UPDATE statement I always explicitly use BEGIN TRAN tomake sure my statement is correct and I get the correct number of results returned.
http://technet.microsoft.com/en-us/library/ms188929.aspxhttp://technet.microsoft.com/en-us/library/ms188929.aspxhttp://technet.microsoft.com/en-us/library/ms188929.aspxhttp://technet.microsoft.com/en-us/library/ms188929.aspx


5/19

Lets say I want to UPDATE the Employee table and set JobTitle equal to 'DBA' where LoginID islike '%barbara%'. I accidentally create my statement wrong and issue the statement belowwhich actually would make every JobTitle equal to 'DBA':

UPDATE HumanResources.Employee

SET JobTitle = DBA

WHERE LoginID IN (

SELECT LoginID FROM HumanResources.Employee)

Oops! I didnt mean to do that!! I accidentally made every record have a JobTitle of DBA. If Iwould have placed a BEGIN TRAN before my statement I would have noticed that 290 resultswould be effected and something is wrong with my statement:


6/19

Since I specified a BEGIN TRAN, the transaction is now waiting on a ROLLBACK or COMMIT.While the transaction is waiting it has created a lock on the table and any other processes thatare trying to access HumanResources.Employee are now being blocked. Be careful using BEGINTRAN and make sure you immediately issue a ROLLBACK or COMMIT:

As you can see, SPID 52 is getting blocked by 54.

Since I noticed something was terribly wrong with my UPDATE statement, I can issue aROLLBACK TRAN statement to rollback the transaction meaning that none of the data actuallychanged:

ROLLBACK TRANSACTION rolls back an explicit or implicit transaction to the beginning of thetransaction, or to a savepoint inside the transaction. It also frees resources held by thetransaction. - MS


7/19

If I had written my statement correct the first time and noticed the right amount of resultsdisplayed then I could issue a COMMIT TRAN and it would execute the statement and mychanges would be committed to the database:

COMMIT TRANSACTION marks the end of a successful implicit or explicit transaction. If@@TRANCOUNT is 1, COMMIT TRANSACTION makes all data modifications performed since thestart of the transaction a permanent part of the database, frees the resources held by thetransaction, and decrements @@TRANCOUNT to 0. If @@TRANCOUNT is greater than 1,COMMIT TRANSACTION decrements @@TRANCOUNT only by 1 and the transaction staysactive. - MS


8/19

What is a nested transaction?

A nested transaction is a transaction that is created inside another transaction.Huh? It soundsconfusing, but its not. A nested transactions purpose is to support transactions in storedprocedures that can be called from a process already in a transaction or from a process that hasno active transaction.See below for an example of a nested transaction:

BEGIN TRAN Tran1GOBEGIN TRAN Nested TranGOINSERT INTO Table1 DEFAULT ValuesGO 10COMMIT TRAN Nested TranSELECT * FROM Table1

I get back 10 results. So far, so good. When I rollback the first transaction. ROLLBACK TRAN Tran1

The 10 transactions are gone. Why? The nested transaction is based on the action of theoutermost transaction. Since I rolled back the outer transaction, the entire transaction is rolledback no matter what I did in between. If I would have committed Tran 1, then my nestedtransaction would have also committed.


9/19

How does Mirroring, Replication, and Log Shipping use the transaction log?

Im not going to go over what each high availability option is, but instead Ill tal k about how thetransaction log is involved with each option.

Database Mirroring - When a DML statement is executed against the primary database,mirroring needs to create this exact same statement on the mirrored database as quickly aspossible. This is done by sending a continuous stream of active transaction log records to themirror server, which applies these logs to the mirrored database, in sequence, as quickly aspossible. Starting in SQL Server 2008, the transaction log record gets compressed beforesending it over to the mirrored server to help reduce latency across the network.

Another consideration when choosing between synchronous or asynchronous mirroring iswhen mirroring is asynchronous, the transactions commit without waiting for the mirror serverto the write the log to the disk, which maximizes performance. In synchronous mirroring, the

transaction is committed on both partners, but that increases network latency.

One thing to take in account with database mirroring is when you pause mirroring, the principaldatabase will still accumulate records in the transaction log and that log file cannot betruncated. Therefore, is the database mirroring stays in the paused state for a long period oftime, it can cause the log to fill up.

Transactional Replication - Transactional replication includes 3 main parts: SQL ServerSnapshot Agent, Log Reader Agent, and Distribution Agent.

During transactional replication the Log Reader Agent monitors the transaction log of the

database replicated and copies the transactions marked for replication into the distributiondatabase (a database created when replication is configured). The Distribution Agent thencomes along and copies the transactions from the distribution database to the Subscriber(replicated database)

The Log Reader Agent runs at the Distributor and when executed it first reads the publicationtransaction log looking for any DML statements. Next, the agent copies those transactionsmarked for replication to the distribution database. The Distribution Agent then moves thetransactions to the Subscriber as described above. Only committed transactions are sent to thedistribution database.

Unlike mirroring, which works at the physical level, replication works at the logical level.

Log Shipping - Theres really not too much involved in log shipping. You can think of it like youthink of Backup/Restore. Log shipping basically consists of backing up a transaction log,shipping (moving) it to another server and restoring the transaction log. The log can also beshipped to multiple secondary servers if needed.


10/19

Why does the transaction log grow?

Like weve already discussed, the transaction log records all database modifications. If you havea busy system, this alone could cause the transaction log to grow. If Autogrow is not enabled,once the log file hits the specified maximum size, it will throw an error on every transaction

that hits the database until the problem is fixed. I always recommend turning on Autogrow, butkeep an eye on your transaction log file. Also, specifying a small autogrowth increment on yourlog file can reduce performance. The file growth increment on a log file should be sufficientlylarge enough to avoid frequent expansion. The default is 10 percent and this may be sufficientfor you environment. I would start at 10 percent, but keep an eye on the log to see how often itgrows and adjust accordingly. Using a fixed increment instead of percentages may work betterfor your environment. There really isnt one setting for all database s, so you need to monitorlog usage to figure out the best setting for each database.

To enable Autogrow in SSMS, right click the database and select properties. Under the

Database properties, choose the Files tab and adjust Autogrowth / Maxsize:


11/19

Other factors that will cause the log file to grow may include the following:

Uncommitted transactions Index Operations - CREATE INDEX, rebuild indexes, etc. Un-replicated transactions

Long running transactions Incorrect recovery model settings (we'll discuss this later) Large transactions

What is a Virtual Log File (VLF)?

Each physical transaction log file is divided internally into numerous virtual log files, or VLFs.The virtual log files are not a certain size nor can you specify how many VLFs are in a physicallog file. The Database Engine does this for us, but for performance reasons it tries to maintain asmall number of virtual files.

System Performance is affected when the virtual log file is defined by small size andgrowth_increment values meaning if the log files grow to a large size because of many smallincrements, it will increase the number of virtual log files. This is why its a good reason to setAutogrow to a larger increment. If the log is set to grow at 1MB at a time it may growcontinuously resulting in more and more virtual log files. An increased number of VLFs can slowdown database startup and log backup/restore operations.

Theres not a right or wrong number of VLFs per database, but remember, the more you have

the worse performance you may have. You can use DBCC LOGINFO to check the number ofVLFs in your database.

Andy Novick wrote an excellent tip on performance issues with a large number of VLFs here.

How to monitor transaction log growth?

Monitoring the log file is very important and SQL Server has made it fairly easy for us to do this.

One way to find information about the log is in the catalog view sys.database_files. This viewreturns information about data and log files that include type of file, name, location, state, size,growth, etc. The following query will filter down to only the log file and displays some veryuseful information:

SELECT name AS [File Name],

physical_name AS [Physical Name],
http://www.mssqltips.com/sqlservertip/2107/performance-issue-with-large-number-of-virtual-log-files-in-sql-server-transaction-log/http://www.mssqltips.com/sqlservertip/2107/performance-issue-with-large-number-of-virtual-log-files-in-sql-server-transaction-log/http://www.mssqltips.com/sqlservertip/2107/performance-issue-with-large-number-of-virtual-log-files-in-sql-server-transaction-log/http://www.mssqltips.com/sqlservertip/2107/performance-issue-with-large-number-of-virtual-log-files-in-sql-server-transaction-log/


12/19

size/128.0 AS [Total Size in MB],

size/128.0 - CAST(FILEPROPERTY(name, 'SpaceUsed') AS int)/128.0 AS [Available Space InMB],

[growth], [file_id]

FROM sys.database_files

WHERE type_desc = 'LOG'

You can also use DBCC SQLPERF (logspace) which has been around for a while. This commanddisplays useful details such as DB name, Log Size (MB) and Log Space Used (%):

The result set would like the following. As you can see there is a lot of great information that isreturned when using LABELONLY.

ColumnName Value MediaName NULLMediaSetId 8825ADE0-2C83-45BD-994C-7469A5DFF124FamilyCount 1FamilySequenceNumber 1MediaFamilyId 8A6648F8-0000-0000-0000-000000000000MediaSequenceNumber 1MediaLabelPresent 0MediaDescription NULLSoftwareName Microsoft SQL ServerSoftwareVendorId 4608MediaDate 02:37.0MirrorCount 1


13/19

How to shrink the transaction log

One thing that I see a lot of administrators ask about is transaction log size and how to truncateit. Log records that are not managed correctly will eventually fill up the disk causing no moremodifications to the database. Transaction log growth can occur for a few different reasons.

Long running transactions, incorrect recovery model configuration and lack of log backups cangrow the log.

Log truncation frees up space in the log file so the transaction log can reuse it. Unless there issome kind of unexpected delay, log truncation will occur automatically after a checkpoint (if thedatabase is in SIMPLE recovery model) or after a log backup (if the database is in FULL or BULK-LOGGED recovery model). MSSQLTips.com offers plenty of tips regarding transaction logtruncation, but Ill show you two ways to shrink the log.

Shrink the log in SQL Server Management Studio

To shrink the log in SSMS, right click the database, choose Tasks, Shrink, Files:

On the Shrink File window, change the File Type to Log. You can also choose to either releaseunused space, reorganize pages before releasing unused space, or empty file by migrating thedata to other files in the same filegroup:


14/19

Shrink the log using TSQL

If the database is in the SIMPLE recovery model you can use the following statement to shrinkthe log file:

DBCC SHRINKFILE (AdventureWorks2012_log, 1)

Replace AdventureWorks2012_log with the logical name of the log file you need shrunk andchange 1 to the number of MB you want the log file shrunk to.

If the database is in FULL recovery model you could set it to SIMPLE, run DBCC SHRINKFILE, andset back to FULL if you dont care about losing the data in the log.


15/19

ALTER DATABASE AdventureWorks2012

SET RECOVERY SIMPLE

GO

DBCC SHRINKFILE (AdventureWorks2012_log, 1)

GO

ALTER DATABASE AdventureWorks2012

SET RECOVERY FULL

**You can find the logical name of the log file by using the following query:

SELECT name FROM sys.master_files WHERE type_desc = 'LOG'

Another option to shrink the log using the FULL recovery model is to backup the log for yourdatabase using the BACKUP LOGstatement and then issue the SHRINKFILE command to shrinkthe transaction log:

BACKUP LOG AdventureWorks2012 TO BackupDevice

How large should the transaction log be?

This is a question Ive seen come up more and more throughout my years as a DBA. How largeshould the transaction log be? Well.it depends.

For normal day to day operations, I would recommend starting at 25% of the data file size. So ifyour data file is 20GB, make the log 5GB. Keep an eye on the log to see if it continuously growsbeyond 5GB. Like we talked about in the VLF section, every time your log grows you add moreVLFs which can cause performance problems. 25% is the recomm ended starting point fornormal operations, however, if youre doing index rebuilds every night or if youre using highavailability options such as replication or mirroring, youll need enough log space to hold ontothe extra transactions that may occur.

25% is just a recommended value to start with. Realistically you should use a test server toexamine how much log space is used at average and peak times and adjust your productionserver leaving a little extra space to spare.
http://www.mssqltips.com/sqlservertutorial/8/sql-server-transaction-log-backups/http://www.mssqltips.com/sqlservertutorial/8/sql-server-transaction-log-backups/


16/19

Do multiple transaction logs help performance?

Multiple transaction log files absolutely does NOT help performance. Multiple data files,however, does help so I think people believe since multiple data files can be beneficial thatmultiple log files are. Not true.

The transaction log is sequential so SQL Server doesnt perform parallel I/Os to the transactionlog if there are multiples. If your system does have multiple log files, the first file is used in itsentirety, then the second file will be used, and so on.

The only time a second transaction log file might be needed is if the first transaction log is fulland the transaction log wont clear, it will then flip over to the second log file allowing databasemodifications until the first log file can clear.

What are the recovery models and how does the log use them?

I mentioned recovery models a few times in the sections above so Ill give a brief overview ofeach one

Full In the Full recovery model, log files should be backed up on a regular basis to prevent thedisk from filling up the drive. Also, in the Full recovery model, you can restore a database to thepoint of failure depending on your log backup schedules. This is the default for the modeldatabase in SQL Server. If you leave this recovery model set to Full, make sure you schedule logbackups.

Simple In the Simple recovery model, transaction log backups cannot be used. This meansthat SQL Server will automatically reclaim disk space at certain intervals, which can be good,but it also means that if a failure were to occur you can only restore back to the last full backupsince all transactions are lost in between. This is generally used in development environments.

Bulk Logged - In the Bulk Logged recovery model, certain large scale or bulk copy operationsare minimally logged. I have never left a database in the Bulk Logged recovery model for a largeamount of time. I usually will have a database in the Full re covery model and if Im doing a bulkinsert and dont need to log the transactions, Ill switch to Bulk Logged and then back to Fullonce this is complete. Bulk Logged, however, does support log backup and point in timerecovery.

Set Recovery Model in SQL Server Management Studio

To change the recovery mode in SSMS, right click the database, choose Properties. On theDatabase Properties window, select Options, Recovery Model:


17/19

Set Recovery Model using TSQL

Use [master]

GO

ALTER DATABASE [AdventureWorks2012] SET RECOVERY SIMPLE WITH NO_WAIT

GO

Replace AdventureWorks2012 with your database name and use SIMPLE, FULL, orBULK_LOGGED for the recovery model.


18/19

How to recover data using a transaction log backup?

When using the Full or Bulk Logged recovery models you can restore to a point in timedepending on how often you create transaction log backups.

Lets say you create a full backup every night at 12:00am and transaction log backups everyhour. Lets now say that the server cra shes at 3:30am and you get an alert and the only way tobring this server back is to restore the data. If you restore the 12:00am full backup, then thetransaction log backups in order from 1:00am, 2:00am, and 3:00am, you will have up to datedata as of 3:00am only losing 30 minutes of data. Of course, you can set transaction log backupsto occur every 5 minutes which makes your point in time of data loss only 5 minutes at themost. The optimal interval depends on factors such as importance of data loss, size of thedatabase, number of transactions, and workload of the server.

Also, when you are restoring transaction logs, you have to restore WITH NORECOVERY until the

last log. When restoring the last log, restore WITH RECOVERY

Its not possible to restore a transaction log backup if:

If a transaction log backup is damaged or missing, you can only restore up to the pointof the damaged or missing log backup. Transaction log backups have to be in sequentialorder.

You restore a database WITH RECOVERY before you are finished restoring all thetransaction logs needed. If this were to happen, you would need to start over from thebeginning with the Full backup.

The following example starts by restoring a full backup followed by two transaction logbackups. Note using the NORECOVERY statement throughout except on the last restore.

RESTORE DATABASE AdventureWorks2012

FROM BackupDevice

WITH NORECOVERY

GO

RESTORE LOG AdventureWorks2012

FROM BackupDevice

WITH FILE = 1, NORECOVERY

GO


19/19

tut - sql server transaction logs

Documents