Hi,
I'm in the process of migrating our MSSQL database to Cassandra database in our DMS. However, the Total remaining time states around 43hours to complete the migration, is this expected for a 200MB database?
I read in the documentation that it takes approximately 1 hour for every 5GB of data.
Thanks,
Carlos
Hi Carlos,
I was involved in a 5 DMA cluster migration. The system had been operational for about 7 years and had some large tables. This system was still running on Windows 2008.
The migration for each DMA took between 5 to 9 days depending upon the size of the database. The databases were in the 300 GB to 700GB range.
When the migration started, the migration tool estimated the completion time at 2 to 4 days but it took longer.
Best Regards,
Randy
Scenario:
Current system: A 5 cluster running 8.5 Dataminer in a 1 to 1 failover mode. So there are 10 DMAs. This system is running on Windows Server 2008.
The goal was to move to new physical servers running VMs, Windows Server 2012, and upgrade to Dataminer 10.
1) What we did was put 8.5 Dataminer on the new physical servers VMs that were not connected to the internet.
2) We did a backup on the operational cluster MSSQL database on each DMA and loaded them it the corresponding new physical servers running 8.5 Dataminer.
3) We then upgrade these new 5 DMAs to 9.0 and then to 9.6.
4) We started Dataminer and connected to the elements as a test the 9.6 upgrade was working. We left the old 8.5 DMA cluster running. Thus we were running in parallel the old 8.5 cluster and the new 9.6 cluster. (note: we had to ” “not” turn up some elements on 9.6 as these elements would not allow more than 1 agent to poll it. ) But this did give us a confidence level check the 9.6 upgrade and new servers were working correctly
5) We started the Cassandra Migration on the 9.6 DMS but it the migration estimate was very long. We shut down the 9.6 Dataminer software to only run the Cassandra Migration by itself with the thought this would speed up the migration. This did not appear to speed up the migration much.
The migration started out with and estimate of 2 or 4 days but it took much longer as I mentioned.
Once the Cassandra migration was complete on the Dataminer 9.6.
1) We then upgraded the new DMS 9.6 to Dataminer 10.
2) We connected the new DataMiner 10 cluster in parallel with the original 8.5 cluster again to soak and compare data. During the soak period we had to make a couple of minor driver changes to a couple of connectors to be Cassandra compliant.
I would note that some trend MSSQL tables where very large as this system has been running for at least 8 years. We discussed with the customer if they needed trend data going back 8 years. They did not need this data and we truncated some of the very large tables to 1 year of data before the Cassandra migration. Example. One trend table had 91 million rows.
Hi Randy,
Thanks for going through this in such great detail! I really appreciate it and gives me a good picture of what to expect with this migration.
Cheers!
Carlos
Hi Carlos,
The current time estimation is based on the total percentage completed within the current time elapsed. From the screenshot, I see that the migration is running for about 13 minutes, with a total percentage of 0.50% completed. If we extrapolate this, it would take 43 hours to reach 100%. So the calculation seems to make some sense.
If I look at how many rows you have in the Info column, in this case around 12000, it seems to take a very long time to migrate them. The migration of Info is at 66%, meaning about 8000 rows have been done in around 13 minutes. This indeed seems very slow.
Perhaps it is interesting to have a look at the load (CPU/memory/disk IO) of the server that is performing the migration.
Best regards,
Simon
Thanks Simon, I will have a look at that
Hi Randy,
Thank you for your response. For this 5 DMA cluster, how was this performed? Did you have to wait for one DMA to finish the migration before the next agent was migrated? During this time, the DMS is fully operational right?
In this case, for our customer, they have a DMA cluster comprised of 6 DM pairs ( 1 standby and 1 active) total of 12 agents. Do you know what would be the best practice for the migration in this scenario?
Thanks a lot!
Carlos