Developer forum

Forum » Development » CommandTimeout on Log Cleanup

CommandTimeout on Log Cleanup

Anders Ebdrup
Anders Ebdrup
Reply

Dear Dynamicweb,

 

We experience that the planned task of "log cleanup" often ends in an exception due to timeout.
I would therefore like to set a command timeout on the scheduled task, and I would like to have a log of which of the table deletions creates an exception.

 

Best regards, Anders


Replies

 
Morten Snedker Dynamicweb Employee
Morten Snedker
Reply

Hi Anders,

The problem you encounter is usually an initial issue: when cleanup has not been performed at all, and you need to clean up the first time. I don't think we will be doing any changes to this, as the issue lies beyond the timeout itself: you also risk a transaction log going wild (disk space and memory), and you overall risk a huge overload.

When we run into solutions with this issue, we choose to handle it manually.

I use

---

SELECT 
    t.NAME AS TableName,
    p.rows AS RowCounter,
    SUM(a.used_pages) * 8 / 1024 / 1024 AS UsedSpaceGB,
SUM(a.used_pages) * 8 / 1024 AS UsedSpaceMB, 
(SUM(a.total_pages) - SUM(a.used_pages)) * 8 / 1024 AS UnusedSpaceMB
FROM 
    sys.tables t
INNER JOIN      
    sys.indexes i ON t.OBJECT_ID = i.object_id
INNER JOIN 
    sys.partitions p ON i.object_id = p.OBJECT_ID AND i.index_id = p.index_id
INNER JOIN 
    sys.allocation_units a ON p.partition_id = a.container_id
LEFT OUTER JOIN 
    sys.schemas s ON t.schema_id = s.schema_id
WHERE 
    t.NAME NOT LIKE 'dt%' 
    AND t.is_ms_shipped = 0
    AND i.OBJECT_ID > 255 
GROUP BY 
    t.Name, s.Name, p.Rows
ORDER BY 
    SUM(a.used_pages) desc

---

to obtain a list of tables, which sorted by size may indicate what needs cleaning up.

With GeneralLog as example:

---

DECLARE @year int
SET @year = 2018 --start this year
 
DECLARE @month int
SET @month = 1 --
 
DECLARE @day int
SET @day = 7 -- leave this alone. Used for deleting in chunks of 7 days to avoid log from exploding
 
WHILE (@year < 2024)
WHILE (@month <= 12)
BEGIN
  WHILE (@day <= 35)
  BEGIN
DELETE FROM GeneralLog
WHERE YEAR(LogDate) = @year
AND MONTH(LogDate) = @month
AND DAY(Logdate) <= @day
 
SET @day = @day + 7
  END --day
  SET @day = 7
  SET @month = @month + 1
END --month
SET @year = @year + 1
END --year
---
 
This will delete in chunks of 7 days, avoiding the overload on log and memory.
 
BR
Snedker
 
Anders Ebdrup
Anders Ebdrup
Reply

Dear Morten,

 

Thanks for your great response.

The problem is that this is not just an one time issue, but an ongoing issue with the amount of data we have in some of our databases (typically the EcomOrderDebugingInfo table).

At least I would like you to consider to have some logging of which tables cannot be cleaned and hopefully also an option to allow the process to clean up data in the other tables than the one that is failed instead of just abandon everyting.

 

Best regards, Anders

 
Morten Snedker Dynamicweb Employee
Morten Snedker
Reply

Hi Anders,,

Makes good sense. I have passed a feature request on your behalf.

BR
Snedker

 

You must be logged in to post in the forum