Hi everyone
Recently we faced an issue when the compaction process was running and stopped working due to this error:
ERROR [CompactionExecutor:15374] 2021-02-03 07:47:06,911 CassandraDaemon.java:217 - Exception in thread Thread[CompactionExecutor:15374,1,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: The paging file is too small for this operation to complete
Is there a recommendation/documentation on how the config should be on the server regarding the paging file? or should be automatically managed?
side note: the configuration on the server was not the default.
Thanks in advance
Hi Jaime,
There could be several reasons why you have this problem.
The ones that we see generally in the field:
- Windows is not able to provide page files fast enough to Cassandra (could be because your disk is not able to cope with the load and therefor not responding fast enough to the OS to change the page files size). To know if this is the case you could evaluate with the Microsoft platform element how your disk is behaving during the time that the error was thrown. If you are running in virtualized environments it also good to perform some benchmarking. Could be that your disk is shared with multiple agents and that when Compaction kicks in on all of them your disk (or connection to the disk) is just not able to handle the load.
Note: that from version 1.1.2.17 of the Microsoft platform you are able to monitor the page file sizes (keep in mind that this can be spread over multiple disks). - If you have very large ssTables (more specific partitions that are spread multiple large ssTables), it could be that Cassandra needs too much memory and that it is exhausting your disk space. If this is the case you should see that the disk space on your machine drops to zero.
Can you have a look if you would be experiencing one of these scenarios?
Hi Jaime,
The following minimum dedicated (meaning minimum at all time) requirements apply at all times for each server:
• 4 KiB blocks should be 10 MB/s or higher.
• 512 KiB blocks should be 300 MB/s or higher.
• The average latency should be lower than 10 ms.
This can also be found on our minimum requirements document:
https://community.dataminer.services/documentation/dataminer-system-requirements/
I think what we have here is more related to the first scenario on virtual machines, due to we don’t see any space problem with the OS drive.
Is there a pass mark requirement that should be accomplished? We’ll start monitoring the page parameters with the new Microsoft driver version, thanks for your comments Michiel