Hi Ryan,

As far as I know, any timeouts or slow queries should be logged (SLDatabase[xxx].txt)

The 30-minute query timeout gets applied by DataMiner for MySQL queries and cannot currently be configured differently.

I guess it could very well be that cleaning the larger table is not a problem because the list of "NOT IN" ids is shorter. The slow query is probably more because of the "NOT IN" list than because of the table size.

As for the listed query with the huge "NOT IN" list: in some scenario’s the list of NOT IN values can grow infinitely over time, eventually breaking the cleanup (which then leaves a table that never gets cleaned out)

Reason is that for every trended table cell, DataMiner always keeps at least one point (the oldest) in the database to be able to display stable values properly. This can give problems for rows that only exist temporarily.

If the cleanup fails, a workaround is to make a scheduled task e.g. every month that runs the exact same query but without the NOT IN values. This will delete all the obsolete points and will reduce the list significantly, however small downside is that if you have a point that never changed in a year and actually still is present in the table in Cube this will result in an empty trend graph (or a warning that there is no trending present, depending on your version)

Example of such a query:

DELETE FROM DATAAVG_40701_3217 WHERE
((iStatus < 0 AND dtFirst < ‘2021-04-03 13:36:46’) OR
(iStatus = 5 AND dtFirst < ‘2022-03-04 13:36:46’) OR
(iStatus = 60 AND dtFirst < ‘2021-04-03 13:36:46’) OR
(iStatus = 120 AND dtFirst < ‘2021-04-03 13:36:46’))

(this is no longer a concern on Cassandra as the cleaning logic works differently)