As part of a cost analysis piece, we’re considering different Dataminer architectures for a few new Dataminer systems to be hosted in AWS. The architecture will utilise the latest Dataminer release updates which is compatible with Amazon Keyspaces and Opensearch.
A line of research is to prototype different architectural variations, the current guide suggest that a Dataminer system is configured against an Opensearch domain. It has been suggested however whether multiple Dataminer systems can be configured to use the same Opensearch cluster/domain.
Is this something which is possible/recommended?
Hi Reza,
Yes it is possible to let multiple Dataminer systems use the same OpenSearch cluster (or Elasticsearch for that matter).
In order to do this you need to define a unique prefix for every Dataminer cluster.
This can be configured via System Center > Database > Database per cluster.
This prefix is then used to construct every index. That way, multiple “duplicate” indices can exist at the same time, one for every Dataminer system.
Take for example the “alarms” index, this exists for every system but to keep them separate you can let one system use prefix DMS1 and another DMS2.
This way 2 indices will be created: “dms1-alarms” and “dms2-alarms” and both can co-exist without interfering with eachother.
The limiting factor will be resources. For Amazon OpenSearch Sevice specifically, the service gives a limit of 1000 shards/node. The more Dataminer systems are connected, the faster you will hit this limit. The number of shards used per DMS is heavily dominated by the use of DOM and SRM, so it’s hard to give an estimation of how many indices each DMS will require.