System database msdb size in huge! How to purge MSDB Backup and Restore History?

 

On one of my SQL Server 2008 Instance, the msdb database sizes around 4 GB.  I decided to find out the which table / index occupies major junk on this. I have used query below;

--- Query 1, if there is partition SELECT TOP 10 SERVERNAME=@@SERVERNAME,DB_NAME=DB_NAME(),TABLE_NAME=OBJECT_NAME(I.ID),INDEX_NAME=I.NAME,INDID,USED, ROWS, SIZE_N_MB = ROUND((USED*8.0/1024.0),2), ROWMODCTR,STATISTICDT=STATS_DATE(I.ID,INDID) FROM SYSINDEXES I, SYSOBJECTS O WHERE I.ID = O.ID AND INDID IN ( 0,1) AND XTYPE = 'U' ORDER BY USED DESC

--- Query 2 If there are partitions SELECT object_name(i.object_id) as objectName, i.[name] as indexName, sum(a.total_pages) as totalPages, sum(a.used_pages) as usedPages, sum(a.data_pages) as dataPages, (sum(a.total_pages) * 8) / 1024 as totalSpaceMB, (sum(a.used_pages) * 8) / 1024 as usedSpaceMB, (sum(a.data_pages) * 8) / 1024 as dataSpaceMB FROM sys.indexes i INNER JOIN sys.partitions p ON i.object_id = p.object_id AND i.index_id = p.index_id INNER JOIN sys.allocation_units a ON p.partition_id = a.container_id GROUP BY i.object_id, i.index_id, i.[name] ORDER BY sum(a.total_pages) DESC, object_name(i.object_id)

The output resulted;

The following 5 big tables takes up 88% of the total database size of 3.8 GB. These 5 tables occupy 3.3 GB on the msdb database.

Query 1: Output

TABLE_NAME	INDEX_NAME	INDID	USED	ROWS	SIZE_N_MB	ROWMODCTR	STATISTICDT
backupfile	PK__backupfi__57D1800A17C286CF	1	233057	4438738	1820.76	2646909	2/14/2011 15:57
backupset	PK__backupse__21F79AAB0E391C95	1	115791	1489763	904.62	228112	6/19/2011 6:00
backupmediafamily	PK__backupme__0C13C86F0880433F	1	50738	1489616	396.39	23386	7/26/2011 7:00
backupfilegroup	PK__backupfi__760CD67A12FDD1B2	1	43284	2967333	338.16	1775145	2/14/2011 15:57
backupmediaset	PK__backupme__DAC69E4D04AFB25B	1	29250	1489616	228.52	44462	7/22/2011 15:43
sysjobhistory	clust	1	3553	33177	27.76	2428	7/25/2011 8:14

Query 2: Output

objectName	indexName	totalPages	usedPages	dataPages	totalSpaceMB	usedSpaceMB	dataSpaceMB	Rows
backupfile	PK__backupfi__57D1800A17C286CF	233073	233043	231653	1820	1820	1809	4438738
backupset	PK__backupse__21F79AAB0E391C95	100217	100200	99826	782	782	779	1489763
backupfilegroup	PK__backupfi__760CD67A12FDD1B2	43289	43283	43077	338	338	336	1489616
backupmediafamily	PK__backupme__0C13C86F0880433F	39473	39459	39287	308	308	306	2967333
backupmediaset	PK__backupme__DAC69E4D04AFB25B	18737	18728	18654	146	146	145	1489616

Analyze and find out the minimum and maximum date range date available

SELECT COUNT(*) AS 'TotalRecords', MIN(backup_start_date) AS 'MinDate', MAX(backup_start_date) AS 'MaxDate' FROM dbo.backupset

Resolution:
The solution is we have to purge these MSDB Backup and Restore history tables

1. The sp_delete_backuphistory system stored procedure takes a single parameter - a cutoff date. Any data older than the supplied date is purged from the msdb tables.

2. The sp_delete_database_backuphistory  system stored procedure allows you to delete historical backup data for a specific database. Unfortunately, this procedure does not offer the finer option of choosing a cutoff date. It's all or nothing.

3. sp_purge_jobhistory Removes the history records for a job.

4. Try considering creating indexes on the above said 5 tables, if no of rows are huge.
Source: http://weblogs.sqlteam.com/geoffh/archive/2008/01/21/MSDB-Performance-Tuning.aspx

use msdb go --backupset Create index IX_backupset_backup_set_id on backupset(backup_set_id) go Create index IX_backupset_backup_set_uuid on backupset(backup_set_uuid) go Create index IX_backupset_media_set_id on backupset(media_set_id) go Create index IX_backupset_backup_finish_date on backupset(backup_finish_date) go Create index IX_backupset_backup_start_date on backupset(backup_start_date) go --backupmediaset Create index IX_backupmediaset_media_set_id on backupmediaset(media_set_id) go --backupfile Create index IX_backupfile_backup_set_id on backupfile(backup_set_id) go --backupmediafamily Create index IX_backupmediafamily_media_set_id on backupmediafamily(media_set_id) go --restorehistory Create index IX_restorehistory_restore_history_id on restorehistory(restore_history_id) go Create index IX_restorehistory_backup_set_id on restorehistory(backup_set_id) go --restorefile Create index IX_restorefile_restore_history_id on restorefile(restore_history_id) go --restorefilegroup Create index IX_restorefilegroup_restore_history_id on restorefilegroup(restore_history_id) go

5. Alternatively, check when the statistics of these tables are last updated. Just run sp_updatestats.  Just try sp_delete_backuphistory  after the statistics are updated. The result is awesome! Very quick response.

Permanent Resolution

6. Further restrict the Job history growth by setting in SQL Agent –> History tab.

7. Also on the SQL Server Maintenance Plan “History Cleanup Task” can be of helpful

SQL Server 2008: syspolicy_purge_history job

 

This job is created automatically when on SQL Server 2008 Policy Management is enabled.

Policy Management is a new feature introduced on SQL Server 2008 onwards.

Once the Policy Management defined, the policy can be evaluated against targets to check for compliance. Each evaluation of compliance will be recorded in tables in system database msdb,

1. syspolicy_policy_execution_history_details_internal
2. syspolicy_policy_execution_history_internal
3. syspolicy_policy_category_subscriptions_internal

The evaluation condition is built on top of object facets, which is predefined by Microsoft.

Purpose of the job, syspolicy_purge_history:

It is to remove records in those tables to maintain so it does not pile up and take up space unnecessarily. It has three steps

It does what instance it is running in and then erases system health records.

Step 1 : 
The first step verifies if Policy Management is enabled. If yes, it will go to the second step
IF (msdb.dbo.fn_syspolicy_is_automation_enabled() != 1)
        BEGIN
            RAISERROR(34022, 16, 1)
        END

step 2
Deletes policy evaluation history prior to cutoff date, defined by HistoryRetentionInDays.
EXEC msdb.dbo.sp_syspolicy_purge_history

step 3
It works out what instance it is running in and then erases system health records.

if ('$(ESCAPE_SQUOTE(INST))' -eq 'MSSQLSERVER') {$a = '\DEFAULT'} ELSE {$a = ''};
(Get-Item SQLSERVER:\SQLPolicy\$(ESCAPE_NONE(SRVR))$a).EraseSystemHealthPhantomRecords()

SQL Server Management Studio – SSMS is a 32 bit application

SQL Server Management Studio – SSMS is a 32 bit application. Even when we install 64 bit SQL Server Database Engine, we will get 32 bit SSMS installed on the server.

Default SSMS install location

32 Bit Server	C:\Program Files\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE
64 Bit Server	C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE

Maintenance Plan Error: Alter failed for Server SQLServer_Instance

 

Error message: Alter failed for Server ‘SivaSQL2008’

 

When I ran SQL Profiler and checked why its failing, observed;
Errors when it was trying to execute

SP_CONFIGURE 'USER OPTION',0 ;
RECONFIGURE

Error: 5808, Severity: 16, State: 1
Ad hoc update to system catalogs is not supported.

The reason for why AD HOC UPDATE TO SYSTEM CATALOGS is not supported.
Why it is trying to change SYSTEM CATALOGS at first place?

In SQL Server 2000 and older, this option allowed users to make updates directly to the system tables.
In SQL Server 2005 and higher, the system tables are, replaced with the Resource Database and system views.
This option is no longer supported in SQL Server 2005 and higher version, and though we can set Allow Updates to 1 with no error. As soon as we run RECONFIGURE, we will receive the error that ad hoc updates are not supported.

Because Allow Updates had been set to 1 and my Maintenance Plan runs the RECONGIFURE statement, this error was thrown and the Maintenance Plan failed.

As per Books On Line:

Changing the allow updates option will cause the RECONFIGURE statement to fail. Changes to the allow updates option should be removed from all scripts.

Resolution:
So to resolve is configured ALLOW UPDATE back to 0 and again ran Maintenance Plan and it executes fine.

sp_configure 'allow update', 0 reconfigure

How to create scripts for an entire database or specific object in SQL Server 2008?

In this post I would like to address the requirement of;

• How to create scripts for an entire database ?
• How to create scripts limit it to specific objects?
• How to restore SQL Server 2005 database on SQL Server 2000?

SQL Server Database Publishing Wizard enables the deployment of SQL Server databases into a hosted environment on either a SQL Server 2000 or 2005 server.

It generates a single SQL script file which can be used to recreate a database (both schema and data) in a shared hosting environment where the only connectivity to a server is through a web-based control panel with a script execution window.

If supported by the hosting service provider, the Database Publishing Wizard can also directly upload databases to servers located at the shared hosting provider.

We can use the Generate and Publish Scripts Wizard to create scripts for transferring a database from one instance of the Database Engine to another. You can generate scripts for a database on an instance of the Database Engine in your local network, or from SQL Azure.

The generated scripts can be run on another instance of the Database Engine or SQL Azure. 

We can also use the wizard to publish the contents of a database directly to a Web service created by using the Database Publishing Services.

We can create scripts for an entire database, or limit it to specific objects.

What version of SQL Servers are supported on this?

	Source Database	Target Database
SQL Server Instance Version Supported	SQL Server 2005 SQL Server 2008 SQL Server 2008 R2 SQL Azure	SQL Server 2005 SQL Server 2008 SQL Server 2008 R2 SQL Azure SQL Server 2000
Permission needed	db_ddladmin	db_ddladmin
Pre-existence		The target database must be created at the hosting provider before the source database is published. Publishing overwrites objects in that existing database

Supported Objects for Publishing

The following table lists the objects that can be published and the versions of SQL Server on which they are supported by the Generate and Publish Scripts Wizard.

Database object	SQL Server 2008 R2	SQL Server 2008	SQL Server 2005	SQL Server 2000
Application role	Yes	Yes	Yes	Yes
Assembly	Yes	Yes	Yes	No
CHECK constraint	Yes	Yes	Yes	Yes
CLR (common language runtime) stored procedure1	Yes	Yes	Yes	No
CLR user-defined function	Yes	Yes	Yes	No
Database role	Yes	Yes	Yes	Yes
DEFAULT constraint	Yes	Yes	Yes	Yes
Full-text catalog	Yes	Yes	Yes	Yes
Index	Yes	Yes	Yes	Yes
Rule	Yes	Yes	Yes	Yes
Schema	Yes	Yes	Yes	No
Stored procedure1	Yes	Yes	Yes	Yes
Synonym	Yes	Yes	Yes	Yes
Table	Yes	Yes	Yes	Yes
User2	Yes	Yes	Yes	Yes
User-defined aggregate	Yes	Yes	Yes	No
User-defined data type	Yes	Yes	Yes	Yes
User-defined function	Yes	Yes	Yes	Yes
User-defined table	Yes	Yes	No	No
User-defined type	Yes	Yes	Yes	No
View1	Yes	Yes	Yes	Yes
XML schema collection	Yes	Yes	Yes

1 Published without encryption.
2 Any non-system users that exist in the database are published as Roles.

How to generate SQL Scripts using Generate and Publish Scripts Wizard

We can create Transact-SQL scripts for multiple objects by using the Generate and Publish Scripts Wizard.. You can also generate a script for individual objects or multiple objects by using the Script as menu in Object Explorer.

Use the Generate and Publish Scripts Wizard to create a Transact-SQL script for many objects.
The wizard generates a script of all the objects in a database, or a subset of the objects that you select. The wizard has many options for your scripts, such as whether to include permissions, collation, constraints, and so on.

Step 1:

In Object Explorer, expand Databases, right-click a database, point to Tasks, and then click Generate Scripts. Follow the steps in the wizard to script the database objects.



 

Step 2: On the Choose Objects page, select the objects to be included in the script

Script entire database and all database objects
Click to generate scripts for all objects in the database and to include a script for the database itself.

Step 1.3: On the Set Scripting Options page, select Save scripts to a specific location.

Select Save scripts to a specific location


Summary:



Click Finish.

To generate the Script for the Data on the table:

Set “Script Data: True “

 

Select specific database objects
Click to limit the wizard to generate scripts for only the specific objects in the database that you choose.

For example I have selected Tables Object

 

Within Tables Object, I have chosen HumanResources.Employee Table

 

Select Save scripts to a specific location



Script Wizard Summary
 

View of the Output file generated:

 

References:

Using the Generate and Publish Scripts Wizard

How to: Generate a Script (SQL Server Management Studio)

Msg 3701, Level 11, State 5, Line 1 Cannot drop the trigger ‘ ‘, because it does not exist or you do not have permission.

 

I got the error message below, when i was playing with DDL Trigger on SQL Server 2008.
The trigger exists, i have cross verified querying;

select * from master.sys.server_triggers where name = 'connection_limit_trigger' select objectproperty(1435152158, 'isTrigger') isTrigger        ,objectproperty(1435152158, 'ownerid') ownerID

Error Message:

Msg 3701, Level 11, State 5, Line 1
Cannot drop the trigger 'connection_limit_trigger', because it does not exist or you do not have permission.

Resolution:

After scratching my head, i found the resolution. It is simple !

If the trigger is DATABASE Level scope use below

Drop TRIGGER connection_limit_trigger on Database

Otherwise if it is Server Level scope please try below;

Drop TRIGGER connection_limit_trigger on all server

Cause:
The lost bold command i.e. scope level makes the different.
When we use DDL Trigger and trying to drop using the DROP command, we should match the scope level of the trigger when it created.

The server principal is set as the execution context of a trigger or event notification and cannot be dropped.

 

I got error message below when i try to drop a login named ‘test_login’

Error Message:
Msg 15186, Level 16, State 1, Line 1
The server principal is set as the execution context of a trigger or event notification and cannot be dropped.

It means that there are  execute as username in a stored procedure or function or triggers, and that is why it will not allow me to drop the user.

EXECUTE permission - a check is made to see if this permission is granted. If yes the user is allowed to run the stored procedure

EXECUTE AS - when the stored procedure is run, it will run under the context of the user specified in the clause rather than the context of the user who has run it.

Use the query below find the execute as object

select object_name(object_id) from sys.sql_modules where execute_as_principal_id = user_id('test_login')

How to find the Dynamic Management Views on SQL Server Management Studio?

 

Expand the Views –> System Views on each database nodes.

Change Tracking Vs. Change Data Capture ( CDC ) in SQL Server 2008

 

Here we find the difference between the Change Track vs Change Data Capture in SQL Server 2008. Both helps in tracking the table changes and security compliance.

Change Track	Change Data Capture – CDC
It is about fact: It captures only the fact as the tracking table has changed. It does NOT capture the data. Therefore, change tracking is more limited in the historical questions it can answer compared to change data capture. However, for those applications that do not require the historical information, there is far less storage overhead because of the changed data not being captured	It is about the Data: Change data capture provides historical change information for a user table by capturing both the fact that DML changes were made and the actual data that was changed.
Storage: Internal tables are placed on the same filegroup as the parent entity. You could use the sys.internal_tables catalog view to show all the internal tables and parent entities. For example: select name, object_name(parent_id) as parent_object from sys.internal_tables	Storage: When change data capture is enabled for a database, a few things are added to the database, including a new schema (called cdc), some metadata tables, and a trigger to capture Data Definition Language (DDL) events. The two function names are, respectively, fn_cdc_get_all_changes_ and fn_cdc_get_net_changes_, with the capture instance name appended. Note that (like the change tracking feature) this functionality requires the table to have a primary key or other unique index.
Supported on “Simple” recovery model also. It is recommended that you use snapshot isolation when change tracking is enabled. Snapshot isolation itself can add significant workload overhead and requires much more careful management of tempdb.	Prevents Log truncation. Forces full logging of some bulk operations. One major point to note here is that once change data capture is enabled, the transaction log behaves just as it does with transactional replication—the log cannot be truncated until the log reader has processed it. This means a checkpoint operation, even in SIMPLE recovery mode, will not truncate the log unless it has already been processed by the log reader.
It uses synchronous tracking mechanism. once a database is enabled for change tracking, a version number is instituted, which allows ordering of operations	Change Data Capture (CDC) uses the asynchronous process that reads the transaction log.
Change Tracking has minimal impact on the system.	It has almost nil impact as it asynchronous mechanism reads from the transaction log.
It uses TempDB heavily	It uses transaction log.
DDL Restriction: There are restrictions on the DDL that can be performed on a table being tracked. The most notable restriction is that the primary key cannot be altered in any way. The other restriction worth calling out here is that an ALTER TABLE SWITCH will fail if either table involved has change tracking enabled.	No such DDL restriction
SQL Agent not needed	It requires SQL Agent to be running. SQL Agent Job & Transaction Replication: Two SQL Agent jobs may be created: the capture job and the cleanup job. I say "may be created" because the capture job is the same as the one used for harvesting transactions in transactional replication. If transactional replication is already configured, then only the cleanup job will be created and the existing log reader job will also be used as the capture job
Permission required to enable: SYSADMIN	Permission required to enable: DBOwner

 

Reference:

Comparing Change Data Capture and Change Tracking

Tracking Changes in your Enterprise

Deny specific user from Remote Login to SQL Server

SQL Server Query to deny user on remote login feature, let user connect to database engine only from localhost (or 127.0.0.1).

This query below applies to SQL Server 2005 with SP2 and above higher version. It uses the SQL Server Logon Trigger

 

CREATE TRIGGER Deny_Remote_Login_Login1_trigger ON ALL SERVER WITH EXECUTE AS 'sa' FOR LOGON AS BEGIN IF ORIGINAL_LOGIN()= 'Login1' AND (SELECT * FROM sys.dm_exec_sessions WHERE is_user_process = 1 AND original_login_name = 'Login1' --- Checking the given login and Host_Name not in ( 'SQLServerHostName') --- Connecting Remotely and Session_ID = @@SPID ) BEGIN PRINT 'RemoteLogin Attempt attempt of ' + ORIGINAL_LOGIN()+' denied, please login locally' ROLLBACK; END END;

How to apply Service pack on SQL Server 2005 Cluster

 

1. Backup of all the System + User Database

2.Windows Login account with "Local Administrator" privileges on all the nodes involved

3. Ensure all the recourses part of the SQL Server Cluster resource group are online.
   I recommend to reboot both nodes before installation. (to avoid pending reboot, hanging installers, terminal sessions etc. problems.)

4. Cross check if the cluster failover on the nodes involved successfully.

5. shut down all applications that use SQLServer (because you know the instance(s) will get offline at some point(s) during the upgrade)

6. Disable if any DDL Triggers

7. Verify if following services online on the nodes involved.

• Task Scheduler
• Com+ System Application Service
• Cryptographic Services
• Remote Registry Service 

8. Pause Database Mirroring

Log on the active node, apply the Service pack. ( It is opposite in case of SQL Server 2008 ! ). You cannot install SP3 from any passive nodes in the failover cluster.

Please note SP will not update the client tools on the passive node. you need to do that manually; though it is cluster aware.

Reference: 

How to install SQL Server 2005 SP3

SQL Server 2005 SP3 Release Notes

SQL Server 2008 failover cluster rolling patch and service pack process

SQL Server 2005 /2008 Express Edition Backup automation

SQL Database Backup Script

declare @IDENT INT, @sql varchar(1000), @DBNAME VARCHAR(200)   select @IDENT=min(DBID) from SYSDATABASES WHERE [DBID] > 0 AND NAME NOT IN ('PUBS', 'NORTHWIND', 'TEMPDB','AdventureWorks') while @IDENT is not null begin     SELECT @DBNAME = NAME FROM SYSDATABASES WHERE DBID = @IDENT /*Change disk location here as required*/     SELECT @SQL = 'BACKUP DATABASE '+@DBNAME+' TO DISK = ''C:\SQLBACKUP\Backup\'+@DBNAME+'.BAK''WITH INIT' PRINT @SQL     EXEC (@SQL)     select @IDENT=min(DBID) from SYSDATABASES WHERE [DBID] > 0 and DBID>@IDENT AND NAME NOT IN ('PUBS', 'NORTHWIND', 'TEMPDB','AdventureWorks') end

 

DOS Batch files to backup the databases

@echo off c: if not exist c:\SQLBackup\Logs MD C:\SQLbackup\Logs if not exist c:\SQLBackup\Backup MD C:\SQLbackup\Backup cd c:\SQLBackup Echo Backup Started >> c:\SQLBackup\Logs\SQLBAckup.log Date /T > c:\SQLBackup\Logs\SQLBAckup.log Time /T >> c:\SQLBackup\Logs\SQLBAckup.log Echo. echo *************************************************************************** >> c:\SQLBackup\Logs\SQLBAckup.log isql -S (local)\Express -E -i c:\sqlbackup\backup.sql >> c:\SQLBackup\Logs\SQLBAckup.log echo *************************************************************************** >> c:\SQLBackup\Logs\SQLBAckup.log echo. Date /T >> c:\SQLBackup\Logs\SQLBAckup.log Time /T >> c:\SQLBackup\Logs\SQLBAckup.log echo Backup Finished >> c:\SQLBackup\Logs\SQLBAckup.log echo. echo *************************************************************************** >> c:\SQLBackup\Logs\SQLBAckup.log

 

Folders will look like,

 
Schedule it on Windows Task Scheduler

That's it the SQL Server 2005 / 2008 Express Edition database backup is automated now!

SQL Server 2008 Service Pack 1 released

SQL Server 2008 Service Pack 1 (SP1) is now available at the following URL :
You can use these packages to upgrade any SQL Server 2008 edition.

SQL Server 2008 SP1 is primarily a roll-up of previous cumulative updates, quick fix engineering updates and minor fixes made in response to requests reported through the SQL Server community. While there are no new features in this service pack, customers running SQL Server 2008 should download and install SP1 to take advantage of the fixes which increase supportability and stability of SQL Server 2008.

With this service pack, Microsoft has made investments to ease deployment and make management of service packs more efficient. Key improvements in SP1 include Slipstream, Service Pack Uninstall and Report Builder 2.0 click-once deployment.
• Slipstream allows administrators to install SQL Server 2008 and Service Pack 1 in a single instance. This decreases the total time for an installation, including a fewer number of reboots thereby increasing productivity and deployment availability.
• Service Pack Uninstall allows administrators to uninstall the service pack separately from the database release. This feature also improves DBA productivity, reduces the cost of deployment and improves overall supportability
• Report Builder 2.0 Click Once improves the existing SQL Server end-user report authoring application by easing deployment to business users

For more information about SQL Server 2008 Service Pack 1, please review the Release Notes :