On the indexing system requirements, the DataMiner System Requirements REV047 are indicating a need for a separate hard disk from the one containing the Cassandra database.
If the customer is completely running on virtual servers, does it still matter if the database is installed on a separate hard disk? Or is the separation in this case more best practice and not mandatory, as long as the disk throughput meets the requirements of above 140 IOPS for 10k RPM SAS disks?
By my understanding, the problem we are trying to address by providing each node with its own hard disk is that no node will be able to hijack all of the IO bandwidth, causing the other node to experience severe performance impact.
If you were to theoretically go for a single disk that had 280 IOPS capabilty and offered enough space to host the dataset of both Cassandra and Elasticsearch combined, you would still have situations where for instance Cassandra is running a repair/compaction action and simply consumes as much IOPS as it can to finish the job. To my knowledge, there's no way you can specify what the max. IOPS bandwidth is that a single node can use. They will simply take what they can get and this will again affect the performance of the other node hosted on the same physical disk.
In order to ensure a minimum set of available IOPS for each node at all time, we specify separate physical hard drives in our requirements.
The same principle also applies in virtualized environments. In there, the VM administrator also needs to be aware that when a virtual disk volume is created, enough IOPS need to be available for that volume at any time and that when one virtual volume fully consumes all of their available IOPS, no other virtual volumes should notice this impact.