We currently have a DMS cluster, each agent has its own single Cassandra node (local or hosted on a separate external server).
For the migration towards Cassandra+OpenSearch Cluster, could we repurpose these external (single Cassandra node) servers specifically for the C+OS cluster?
In theory it sounds simple, reduce the resources of the current external VMs and with those resources create a new Linux VMs to be used for the C+OS cluster
However I'm not sure if this is a good way, if it's feasible or what kind of issues to expect.
I'm currently thinking of:
- Some downtime is expected to reduce the resources of the current VMs
- We should be vigilant when reducing the resources (compaction > requires extra disk space, page file > memory,...)
- It'll also depend on the physical hardware capacities
Hi Robin, This one is a bit difficult to predict and to be sure we need to analyze the exact situation your are in.
During the migration, both (current + new clusters) will be under some stress. Therefore reducing the resources (even though temporary) can get problematic.
For this reason i would say that in general this is definitely not standard procedure.
Do you have any metrics on the amount of data which needs to be migrated? Because this will likely be the biggest factor in determining if the approach with reducing the resources is feasible or not.
Hi Robin,
Think it will highly depend on the use-case and the downtime that is allowed to do the migration. From this question it is hard to understand the scale of the problem.
Not sure how far you need to go (how far the machines are running on the limits). But if you are unable to spin up extra servers during the migration (insufficient resources), you could stop the DMA's to free up resources and use some of those temp to spin up new VMs to do the migration. Once the migration is finished you can stop the old Cassandra nodes and allocate more resources again to your DMA's before you start DM again.
Maybe you could also safe sufficient resources by changing some config on Cassandra temp (e.g. changing gc_grace_seconds, disabling compaction on read, reducing replication factor and doing some cleanup on DB => might be able to save some nodes by reducing the RF/redundancy, ...) or stopping DM to avoid ingesting new data.