sql automation 20090610

Using Automation to Simplify SQL Server Management

Greg RobidouxEdgewood Solutionsgregr@edgewoodsolutions.com

Bullet ManaleIdera

www.idera.com

Performance & Availability

Backup & Recovery

Compliance & Security

Change Management

Idera Solutions for SQL Server

Administration

3

Agenda Why Automate What to Automate How to Automate

◦ SQL Server Tools◦ OtherTools

Questions / Wrap Up

4

Why automateRedundant tasksOnly want to know when there

are issuesAutomatic recoveryReduce manual stepsCreate a repeatable process

5

What to automate Backups Integrity Checks Index Maintenance Link Server Status Free Disk Space Transaction Log Size Replication Alerts Database Mirroring Alerts Reading SQL Server Error Log Scripts to Create Scripts High Availability Gathering Performance Statistics

◦ Trace◦ Perfmon◦ DMVs

6

What to collectBackups – failed and successfulReplication ErrorsMaintenance TasksFailed LoginsSQL Server ErrorsServer Status

7

ProcessSetup

◦ Setup key components such as Database Mail, Operators, etc…

Scripts◦ Create scripts or sets of code to gather data

Automate◦ Schedule process to run and collect data. This could

be something you schedule or something that occurs automatically within SQL Server.

Analyze◦ Analyze data and determine what to do.

Notify◦ Send notification to either a person or another

process.

8

NotificationsDatabase MailSQL Agent NotificationsAlertsOperatorsOther Tools

9

Database MailUses SMTPSetup Default Profilesp_send_dbmail

http://www.mssqltips.com/tip.asp?tip=1736

http://www.mssqltips.com/tip.asp?tip=1100

http://www.mssqltips.com/tip.asp?tip=1261

10

SQL Agent Alert SystemTo send out notifications for

scheduled jobs you need to enable a mail profile.

11

AutomateOptions

◦Maintenance Plans◦Scripting

T-SQL PowerShell Windows Script SQLCMD SSIS Etc…

Scheduling

12

Maintenance PlansAllows you to automate routine

tasks such as: ◦Backups◦Index maintenance◦Integrity checks◦Cleanup tasks

Creates SSIS PackagesLimited control

13

Maintenance Plan TasksBackup Database TaskCheck Database Integrity TaskExecute SQL Server Agent TaskExecute T-SQL Statement TaskHistory Cleanup TaskMaintenance Cleanup TaskNotify Operator TaskRebuild Index TaskReorganize Index TaskShrink Database TaskUpdate Statistics Task

14

Maintenance PlansMaintenance Plans Use

◦ master.dbo.xp_create_subdir◦ msdb.dbo.sp_delete_backuphistory◦ msdb.dbo.sp_purge_jobhistory◦ msdb.dbo.sp_maintplan_delete_log◦ master.dbo.xp_delete_file◦ msdb.dbo.sp_notify_operator◦ BACKUP…◦ ALTER INDEX…◦ DBCC CHECKDB◦ DBCC SHRINKDATABASE◦ msdb.dbo.sp_start_job◦ UPDATE STATISTICS

15

Maintenance Plans – Other OptionsSubplansReportingScheduling

16

ScriptingMore work, but gives you more

control.Stored Procedures (T-SQL or CLR)SSIS PackagesVBScriptVB.Net or C#PowerShellSMO (SQL Management Objects)SQLCMDUse of DMVs

17

Scripting ExamplesIn addition to Maintenance Plans

◦Check Free Disk Space◦Transaction Log Usage◦Last Backup Info◦Reading Error Logs◦Selectively Rebuild or Reorganize Indexes◦Scheduled Job Failures◦Backup Failures◦Login Failures◦Scripts that Generate Scripts

18

Backup All Databases Script

DECLARE @name VARCHAR(50) -- database name  DECLARE @path VARCHAR(256) -- path for backup files  DECLARE @fileName VARCHAR(256) -- filename for backup  DECLARE @fileDate VARCHAR(20) -- used for file name

SET @path = 'C:\Backup\' 

SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

DECLARE db_cursor CURSOR FOR  SELECT name FROM master.dbo.sysdatabases WHERE name NOT IN ('master','model','msdb','tempdb') 

OPEN db_cursor   FETCH NEXT FROM db_cursor INTO @name  

WHILE @@FETCH_STATUS = 0   BEGIN          SET @fileName = @path + @name + '_' + @fileDate + '.BAK'         BACKUP DATABASE @name TO DISK = @fileName 

       FETCH NEXT FROM db_cursor INTO @name   END  

CLOSE db_cursor   DEALLOCATE db_cursor

http://www.mssqltips.com/tip.asp?tip=1070

19

Log Space Usage ScriptCREATE TABLE ##logspace

(databaseName sysname,

logsize decimal(10,5),

logused decimal(10,5),

status int)

INSERT INTO ##logspace

EXEC ('dbcc sqlperf(logspace)')

EXEC msdb.dbo.sp_send_dbmail

@profile_name = 'SQLMail Profile',

@recipients = 'gregr@edgewoodsolutions.com',

@query = 'SELECT * FROM ##logspace WHERE logused > 75' ,

@subject = 'Log Space Usage‘

DROP TABLE ##logspace

20

Free Drive Space ScriptUse sys.xp_fixeddrives

CREATE TABLE #drivespace

(drive varchar(20),

freespace bigint)

INSERT INTO #drivespace

EXEC sys.xp_fixeddrives

SELECT * FROM #drivespace

WHERE freespace < 10000

DROP TABLE #drivespace

21

Scripts that Generate Scripts

Use system Meta DataStored proceduresIndex drops and creationsConstraints drops and creationsCreate insert statementsSSMS Scripting

http://www.mssqltips.com/tip.asp?tip=1376

http://vyaskn.tripod.com/code.htm#inserts

22

Reading Error LogsType of System Logs

◦Database Mail◦SQL Agent◦SQL Server◦Windows

23

SP_ReadErrorLogEXEC sp_readerrorlogParameters

1. Value of error log file you want to read: 1. 0 = current, 1 = Archive #1, 2 = Archive #2,

etc...

2. Log file type: 1. 1 or NULL = error log, 2 = SQL Agent log

3. Search string 1: 1. String one you want to search for

4. Search string 2: 1. String two you want to search for to further refine

the results

EXEC sp_readerrorlog 0, 1, ‘Error’

24

XP_ReadErrorLog

EXEC xp_readerrorlog Parameters

1. Value of error log file you want to read: 0 = current, 1 = Archive #1, 2 = Archive #2, etc...

2. Log file type: 1 or NULL = error log, 2 = SQL Agent log 3. Search string 1: String one you want to search for 4. Search string 2: String two you want to search for to

further refine the results5. Search from  start time 6. Search to end time7. Sort order for results: N'asc' = ascending, N'desc' =

descending

http://www.mssqltips.com/tip.asp?tip=1476

http://www.mssqltips.com/tip.asp?tip=1735

25

Scripting

Another way to read the error logs is to read line by line and searching for keywords.

Can be done using any programming language.

This tip shows how it can be done using Windows Scripting http://www.mssqltips.com/tip.asp?tip=1307

Another tool is the Log Parser tool from Microsoft◦http://

www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

26

SchedulingSQL AgentWindows Scheduled TasksVisualCronOther Third Party ToolsSQL Express – does not include

SQL Agent

27

SQL AgentCan Run

◦ActiveX, CmdExec, Replication, SSAS, SSIS, T-SQL

Run when SQL Agent startsSQL Agent Alert Systemsp_start_jobSystem TablesNotificationsLoggingAlerts Can Start Job

28

Scheduling – FrequencyBackups - DailyIntegrity Checks - WeeklyIndex Maintenance - WeeklyOther Tasks – DependsReports – Daily, Weekly

29

Multi Server Administration

Master / TargetManage all scheduled jobs from

one server

30

AlertsReplicationDatabase MirroringBackupsUser DefinedEtc…Response

◦Execute a Job◦Notify Operator

http://www.mssqltips.com/tip.asp?tip=939

http://www.mssqltips.com/tip.asp?tip=1564

31

Central Management ServerNew in SQL 2008Allows you to register servers and

fire off same query on all serversUses Windows Authentication OnlyCan be used for SQL 2000, 2005

and 2008

http://www.mssqltips.com/tip.asp?tip=1767

32

Other ToolsTracePerfmonDMVs

http://www.mssqltips.com/tip.asp?tip=1035

http://www.mssqltips.com/tip.asp?tip=1211

http://www.mssqltips.com/tip.asp?tip=1722

http://www.mssqltips.com/category.asp?catid=31

33

Startup Procedures

USE MASTERGOEXEC SP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', 'ON'GO

USE MASTERGOEXEC SP_PROCOPTION ‘SP_LOG_SERVER_START’, 'STARTUP', ‘OFF'GO

http://www.mssqltips.com/tip.asp?tip=1574

Questions and Wrap-up

• Thanks to our sponsor: Idera• Next webcast in the series:

– “Under the Hood with SQL Server Fundamentals”– Don Jones– July 8th, 2009, 4pm EDT– https://www2.gotomeeting.com/register/449103978

• Download SQL diagnostic manager