How does DM with Cassandra deal with trend data that does not change for long periods of time? In MySQL based systems the DM db cleanup task issues queries on every trend table searching all older outdated data to be deleted looking for entries with the MAX id. It then uses the response from these queries to NOT delete these items to ensure that there is always at least 1 old data point for all entries to Plot in trend graphs. This ensures that trend graphs can plot data points that have not changed since the cut off time to keep that data. In large tables with lots of parameters being trended this can result in Delete Statements that contain an excessive number of table entries IDs to NOT delete. In all of our larger systems this is a serious problem because the Deletes statements contain a list of 50K to 70K ids in the NOT IN() portion of the delete statement. Yes, 50 to 70 thousand ids in a single SQL Delete statement. I understand that moving to Cassandra should resolve part of the problem since DM no longer needs to handle deleting older items from the database since it will be handled natively by the Cassandra using the TTL field, however how does DM/Cassandra handle not deleting older items that have not changed. I am concerned that Cassandra based systems may still have issues with datasets that cause the above mentioned MySQL cleaning behavior? What size datasets have been used in testing Cassandra, anything close to what would result in the above mentioned MySQL Delete behavior?

Thanks in advance.