In a Cassandra based DM failover configuration, how can one tell if the 2 Cassandra node are %100 in sync, contain the exact same data? My understanding is that the nodetool status command’s “Load” column value for each node will match if they are in sync but mine never match and I have seen in them be considerably different, sometimes close to a factor of 2. For example currently it is showing DMA 1 = 1.38G and DMA 2 = 76.92M. This is a small test system so not a large database but that difference seams to be an indication that the 2 nodes do not contain the same data.
Thanks
Hi Jeff we recentyl recieved a similar question and this was the Cassandra champions answer:
Having a large size difference could be normal as it highly depends on when the repair/compaction actions happen.
To give you an example you can see below graph.
When a compaction is performed (should be triggered on both nodes on different time) you will see the live space used going down.
When you perform a repair it could be that the size increases (if not all data was on both nodes).
If you would have a lot of RealTime trending (data kept for 1 day by default) you will see large drops when performing compaction.
That is hen you could have larger differences than 10% which could be perfectly normal.
I would say that it is more important to check if there are no hint files created and if there are no repairs failing.
Looks like Jeroen also posted a response but it is no longer visible. In response to his post I received by email, my test system is a basic standard DM 10.1 CU07 failover configuration running Cassandra on both DMAs, not dedicated dbase servers. Nodetool status is still showing a similar difference in Load values this morning. I have never run the repair operation due to not seeing a need to. I am not aware of any conditions, such as a DMA going offline for any period of time, that could account for this difference and trigger the need to run repair
I just ran the nodetool repair operation and the status still returns a significant difference in the Loads. DMA 1 = 274M DMA 2=1.48G. Shouldn’t these match almost exactly after the repair? The repair operation returned with “completed successfully”
As this question has been inactive for a long time, we will now close it. If you still want further information, could you post a new question?