We have a dedicated 4 node cluster spread across 2 DCs, with a RF=1 at each DC. The consistency level is local quorum. In the event of an outage where a single node at the local DC goes, we understand that approx. 50% of the data will not be readable. However for writes is this data offloaded locally on to the DataMiner agent to be written when the node comes back up or is it simply lost?
Hi Oscar,
When writing data (same for reading data) towards Cassandra you need to define a CL (Consistency Level). This defines how many nodes need to reply (that hold the data) before the command is considered successful. The RF (Replication Factor) defines how many nodes hold a copy of your data. The RF should be configured for every DC through the NetworkTopologyStrategy.
This means that if your configuration is RF1 for every DC by using NetworkTopologyStrategy and you are using CL ONE (only one node that is responsible for the data needs to reply), then if all nodes in one DC are up, then the query should succeed. And therefore DataMiner will consider the Data being written to DB. If the query fails because no node can answer, then DataMiner will go into offload mode (storing all data for DB in files until the DB comes up again to send the data).
If some of the Cassandra nodes were offline, but the data was written to DB successfully (meeting the CL), then it will depend on how long your nodes were offline on how the data can be restored to those nodes. If the node was only offline for a brief moment, then Cassandra will use Hint files to keep data of nodes being marked as just down and then push those files when they come up again. If your nodes are offline for a longer period of time, then a repair will be needed to get the data restored.
Your data consistency will depend on your RF and CL. If your read CL and write CL together is greater than your RF, then you will always have the latest data. This tool might help you to calculate this.
You can find more information in our docs:
Data replication and consistency configuration | DataMiner Docs
or on the Cassandra website:
Welcome to Apache Cassandra’s documentation! | Apache Cassandra Documentation