In order to generate execution plans, SQL Server makes estimations on number of rows to be affected by different operators. But sometimes even your query is simple enough and statistics are up-to-date, SQL Server may still makes bad estimation which is quite far away from the actual number of rows, especially if your table data is skewed (row distribution is not even) and you pass variable/parameter into the query (which is very common in stored procedures).
Let's see an example below:
DECLARE @ID int = (SELECT TOP 1 ID FROM CSAccount WHERE Username = 'awe001');
SELECT ID FROM CSAccount WHERE Parent = @ID;
In this example, the CSAccount table is skewed, most Parent values only have a few rows, but Parent 'awe001' have 128 rows.
Check the execution plan of the 2nd query, the Estimated Number of Rows and the Actual one are very different (2.95 vs 128).
It is because from the query optimizer perspective, it does not know the value of the @ID variable when the query is compiled and before query is executed, values of variables can only be determined during run time. In order to let the optimizer to use the run time variable value for query compilation, we can specify a query hint RECOMPILE. When compiling query plans, the RECOMPILE query hint uses the current
values of any local variables in the query and, if the query is inside a
stored procedure, the current values passed to any parameters. RECOMPILE
is a useful alternative to creating a stored procedure that uses the
WITH RECOMPILE clause when only a subset of queries inside the stored
procedure, instead of the whole stored procedure, must be recompiled.
Let's see the effect of RECOMPILE query hint:
DECLARE @ID int = (SELECT TOP 1 ID FROM CSAccount WHERE Username = 'awe001');
SELECT ID FROM CSAccount WHERE Parent = @ID OPTION (RECOMPILE);
Now the estimation becomes accurate.
Accurate cardinality estimation plays an important role on generating execution plan. The whole structure of the execution plan will have huge difference if the estimation is far away, which can make the query runs very slow. So when you find a query/stored procedure runs slow, especially with a specific set of parameters, then you should check the actual execution plan of it, try to find any edges/operators inside the plan get bad cardinality estimations. If the table data is also skewed, then the query hint RECOMPILE may help.
2016-01-05
2015-12-10
Speedup Query by Indexed, Persisted, Computed Column
Some predicates on table columns in a query for result filtering (WHERE clause) or joining (JOIN... ON clause) cannot be easily resolved back into the form of (raw column = 'xxx'), which hinders the query able to be covered by an index seek operator. For example, WHERE ISNULL(col1, 0) = @var, this predicate cannot be simply resolved, except WHERE (col1 = @var OR (col1 IS NULL AND @var = 0), but such statement won't have good performance.
In order to solve this problem, we can add a PERSISTED COMPUTED column on that table, e.g. ADD col1NullToZero AS ISNULL(col1, 0) PERSISTED. By making the computed column as persisted, we can create indexes on it (ref. Creating Indexes on Persisted Computed Columns). Then you can have an index able to cover the query. Surprise that even the query doesn't directly specify the computed column in its predicate, SQL Server still able to discover the benefit of using the index on the computed column.
Let's see an example below:
1. Create a table, called [MainTran], with two columns: TranID as its primary key, and a Nullable column DepositID that has an index.
CREATE TABLE [MainTran] (
[TranID] [int] NOT NULL PRIMARY KEY,
[DepositID] [int] NULL
)
GO
CREATE NONCLUSTERED INDEX [IX_MainTran_DepositID] ON [MainTran] (
[DepositID]
)
GO
2. Populate some rows into it. Some rows with concrete DepositID values, some DepositID are 0, some DepositID are NULLs.
3. The following query can be fulfilled by index seek, but the result is not correct:
4. The following query result is correct, but it's scan the whole table:
5. The following query gets more complicated execution plan and worse performance:
6. Create a PERSISTED COMPUTED column based on that filtering column, and add an index on it:
ALTER TABLE dbo.MainTran ADD DepositIdNullToZero AS ISNULL(DepositID, 0) PERSISTED
GO
CREATE INDEX IX_MainTran_DepositIdNullToZero ON MainTran (DepositIdNullToZero);
GO
7. Using the Computed column on the query, now it uses seek on the new index:
8. Even the query is using the original expression predicate, SQL Server still able to use the new index on the computed column!
In order to solve this problem, we can add a PERSISTED COMPUTED column on that table, e.g. ADD col1NullToZero AS ISNULL(col1, 0) PERSISTED. By making the computed column as persisted, we can create indexes on it (ref. Creating Indexes on Persisted Computed Columns). Then you can have an index able to cover the query. Surprise that even the query doesn't directly specify the computed column in its predicate, SQL Server still able to discover the benefit of using the index on the computed column.
Let's see an example below:
1. Create a table, called [MainTran], with two columns: TranID as its primary key, and a Nullable column DepositID that has an index.
CREATE TABLE [MainTran] (
[TranID] [int] NOT NULL PRIMARY KEY,
[DepositID] [int] NULL
)
GO
CREATE NONCLUSTERED INDEX [IX_MainTran_DepositID] ON [MainTran] (
[DepositID]
)
GO
2. Populate some rows into it. Some rows with concrete DepositID values, some DepositID are 0, some DepositID are NULLs.
3. The following query can be fulfilled by index seek, but the result is not correct:
4. The following query result is correct, but it's scan the whole table:
5. The following query gets more complicated execution plan and worse performance:
6. Create a PERSISTED COMPUTED column based on that filtering column, and add an index on it:
ALTER TABLE dbo.MainTran ADD DepositIdNullToZero AS ISNULL(DepositID, 0) PERSISTED
GO
CREATE INDEX IX_MainTran_DepositIdNullToZero ON MainTran (DepositIdNullToZero);
GO
7. Using the Computed column on the query, now it uses seek on the new index:
8. Even the query is using the original expression predicate, SQL Server still able to use the new index on the computed column!
2015-11-13
Recover Log Shipping Secondary DB from status Suspect
Sometimes your Log Shipping Secondary database will be go into "Suspect" mode, due to various reasons including log shipping Restore Job failed, log backup file corrupted, and transaction log backup files grown too large due to maintenance task on primary database such as index rebuild. You can recover the secondary database by restoring a recently Full/Differential database Backup WITH NORECOVERY option. Below are the steps:
- Make a Full/Differential database Backup from the Primary database;
- Copy this database backup to the Secondary server;
- Restore the Secondary database by this backup WITH NORECOVERY;
- Start the Log Shipping Restore Job on Secondary server;
- Check the Job History of the Log Shipping Restore Job, you can see it skips all the log backup files with LSN before the database backup you just restored;
- If the Log Shipping Copy Job is still running, eventually the Restore Job will find the log backup files with LSN after the database backup;
- At last the secondary database catch up and resumes.
2015-11-03
Checking Log Shipping Performance
Below SQL script can be used to check the performance of Log Shipping, including the log backup size and time of delay between the primary database and secondary database.
DECLARE @dbname sysname, @days int
SET @dbname = 'SalonWebDB'
SET @days = -14 --previous number of days, script will default to 30
SELECT
rsh.destination_database_name AS [Database],
CASE WHEN rsh.restore_type = 'D' THEN 'Database'
WHEN rsh.restore_type = 'F' THEN 'File'
WHEN rsh.restore_type = 'G' THEN 'Filegroup'
WHEN rsh.restore_type = 'I' THEN 'Differential'
WHEN rsh.restore_type = 'L' THEN 'Log'
WHEN rsh.restore_type = 'V' THEN 'Verifyonly'
WHEN rsh.restore_type = 'R' THEN 'Revert'
ELSE rsh.restore_type
END AS [Restore Type],
rsh.restore_date AS [Restore Date],
bmf.physical_device_name AS [Restored From],
bs.backup_start_date,
bs.backup_finish_date,
CAST(bs.backup_size / 1024.0 / 1024.0 AS decimal(19, 2)) AS [Backup Size MB],
DATEDIFF(second, bs.backup_finish_date, rsh.restore_date) AS [Delay in sec.]
FROM msdb.dbo.restorehistory rsh
INNER JOIN msdb.dbo.backupset bs ON rsh.backup_set_id = bs.backup_set_id
INNER JOIN msdb.dbo.restorefile rf ON rsh.restore_history_id = rf.restore_history_id
INNER JOIN msdb.dbo.backupmediafamily bmf ON bmf.media_set_id = bs.media_set_id
WHERE rsh.restore_date >= DATEADD(dd, ISNULL(@days, -30), GETDATE()) --want to search for previous days
AND destination_database_name = ISNULL(@dbname, destination_database_name) --if no dbname, then return all
AND rsh.restore_type = 'L' -- Log
ORDER BY rsh.restore_history_id
Using the query result, you can plot a graph to see the trends of size and delay.
DECLARE @dbname sysname, @days int
SET @dbname = 'SalonWebDB'
SET @days = -14 --previous number of days, script will default to 30
SELECT
rsh.destination_database_name AS [Database],
CASE WHEN rsh.restore_type = 'D' THEN 'Database'
WHEN rsh.restore_type = 'F' THEN 'File'
WHEN rsh.restore_type = 'G' THEN 'Filegroup'
WHEN rsh.restore_type = 'I' THEN 'Differential'
WHEN rsh.restore_type = 'L' THEN 'Log'
WHEN rsh.restore_type = 'V' THEN 'Verifyonly'
WHEN rsh.restore_type = 'R' THEN 'Revert'
ELSE rsh.restore_type
END AS [Restore Type],
rsh.restore_date AS [Restore Date],
bmf.physical_device_name AS [Restored From],
bs.backup_start_date,
bs.backup_finish_date,
CAST(bs.backup_size / 1024.0 / 1024.0 AS decimal(19, 2)) AS [Backup Size MB],
DATEDIFF(second, bs.backup_finish_date, rsh.restore_date) AS [Delay in sec.]
FROM msdb.dbo.restorehistory rsh
INNER JOIN msdb.dbo.backupset bs ON rsh.backup_set_id = bs.backup_set_id
INNER JOIN msdb.dbo.restorefile rf ON rsh.restore_history_id = rf.restore_history_id
INNER JOIN msdb.dbo.backupmediafamily bmf ON bmf.media_set_id = bs.media_set_id
WHERE rsh.restore_date >= DATEADD(dd, ISNULL(@days, -30), GETDATE()) --want to search for previous days
AND destination_database_name = ISNULL(@dbname, destination_database_name) --if no dbname, then return all
AND rsh.restore_type = 'L' -- Log
ORDER BY rsh.restore_history_id
Sample result:
2015-10-09
Offload Readonly query to Log Shipping Secondary server
You can reduce the load on your primary/production server by using a Log Shipping secondary
server for read-only query. To do this, the secondary
database must be in STANDBY mode. You will not be able to run queries if
the database is in NORECOVERY mode. When you configure log shipping secondary server using SSMS, on the "Restore Transaction Log" tab, choose the "Standby mode" option. You can also decide whether to disconnect users from the secondary database while the restore operation is underway.
After the log shipping secondary server setup is completed, you can check that the secondary database is in standby/readonly mode:
Now you are able to run readonly query on the secondary database:
If you try modifying data on the secondary database, it will be fail:
If you did NOT enabled the "Disconnect users in the database when restoring backups" option, then the Restore Job cannot restore transaction log backups to the secondary database if there are users connected to that database. Transaction log backups will accumulate until there are no user connections to the database.
After all the connections to the secondary database are disconnected, the restore job will be resumed.
Any data modifications in the primary database will be transferred to the secondary database.
Trick: in order to prevent the readonly query blocks the restore job, you can specify the connection to USE another database, e.g. master, and query the secondary database tables using three-part-name.
Now you are able to run readonly query on the secondary database:
If you try modifying data on the secondary database, it will be fail:
If you did NOT enabled the "Disconnect users in the database when restoring backups" option, then the Restore Job cannot restore transaction log backups to the secondary database if there are users connected to that database. Transaction log backups will accumulate until there are no user connections to the database.
Any data modifications in the primary database will be transferred to the secondary database.
Trick: in order to prevent the readonly query blocks the restore job, you can specify the connection to USE another database, e.g. master, and query the secondary database tables using three-part-name.
2015-09-20
Check the Progress of Shrink Database
You can use SSMS GUI or DBCC SHRINK command to shrink the size of the data and log files in a specified database. By the way, both the SSMS GUI and DBCC command do NOT tell you the progress of it. You can consult the sys.dm_exec_requests DMV's percent_complete column to check the progress as demonstrated below:
SELECT percent_complete, start_time, [status], command, estimated_completion_time, cpu_time, total_elapsed_time FROM sys.dm_exec_requests WHERE percent_complete > 0
You should look for DbccFilesCompact in the command column.
SELECT percent_complete, start_time, [status], command, estimated_completion_time, cpu_time, total_elapsed_time FROM sys.dm_exec_requests WHERE percent_complete > 0
You should look for DbccFilesCompact in the command column.
2015-08-26
View Running SQL Statement, inside current processing Batches and Stored Procedures
Below SQL script can be used to view the current running (running, in runnable queue, rolling-back, and waiting for resources like locks) SQL statement, inside every current processing batches and stored procedures. Noted that the [text255] column represents the whole batch/stored procedure, but only shows the first 255 characters. Also, stored procedures are represented as "CREATE PROCEDURE" statement, it's just the definition, does NOT mean it's creating it.
This script is very useful for checking the SQL statement being stuck inside a batch and stored procedure.
SELECT
CASE WHEN (SELECT COUNT(*) FROM sys.sysprocesses WHERE spid = r.spid) > 1 THEN 'Multithread' ELSE '' END AS Multithread,
LEFT(t.[text], 255) AS [text255],
SUBSTRING(
t.[text],
(r.stmt_start / 2) + 1,
((
CASE
r.stmt_end
WHEN -1
THEN DATALENGTH(t.[text])
ELSE r.stmt_end
END - r.stmt_start) / 2) + 1) AS stmt,
DB_NAME(t.dbid) AS ObjectDB,
OBJECT_NAME(t.objectid, t.dbid) AS Object,
r.spid, r.ecid, r.blocked, r.waittime, r.lastwaittype, r.waitresource, DB_NAME(r.dbid) AS connectDB, r.cpu, r.physical_io, r.memusage, r.login_time,
r.last_batch, r.open_tran, r.status, r.hostname, r.program_name, r.loginame, r.cmd, r.net_library, r.login_time, r.stmt_start, r.stmt_end
FROM sys.sysprocesses AS r
CROSS APPLY sys.dm_exec_sql_text(r.[sql_handle]) AS t
WHERE r.status IN ('runnable', 'suspended', 'running', 'rollback', 'pending', 'spinloop')
ORDER BY spid, ecid
This script is very useful for checking the SQL statement being stuck inside a batch and stored procedure.
SELECT
CASE WHEN (SELECT COUNT(*) FROM sys.sysprocesses WHERE spid = r.spid) > 1 THEN 'Multithread' ELSE '' END AS Multithread,
LEFT(t.[text], 255) AS [text255],
SUBSTRING(
t.[text],
(r.stmt_start / 2) + 1,
((
CASE
r.stmt_end
WHEN -1
THEN DATALENGTH(t.[text])
ELSE r.stmt_end
END - r.stmt_start) / 2) + 1) AS stmt,
DB_NAME(t.dbid) AS ObjectDB,
OBJECT_NAME(t.objectid, t.dbid) AS Object,
r.spid, r.ecid, r.blocked, r.waittime, r.lastwaittype, r.waitresource, DB_NAME(r.dbid) AS connectDB, r.cpu, r.physical_io, r.memusage, r.login_time,
r.last_batch, r.open_tran, r.status, r.hostname, r.program_name, r.loginame, r.cmd, r.net_library, r.login_time, r.stmt_start, r.stmt_end
FROM sys.sysprocesses AS r
CROSS APPLY sys.dm_exec_sql_text(r.[sql_handle]) AS t
WHERE r.status IN ('runnable', 'suspended', 'running', 'rollback', 'pending', 'spinloop')
ORDER BY spid, ecid
Subscribe to:
Posts (Atom)

















