Hi,
I'm helping a user with High-availability testing a DMA cluster against an Elastic cluster.
The entry in db.xml on each DMA is as follows:
<DBServer>DMA-elastic1.fqdn.server,DMA-elastic2.fqdn.server,DMA-elastic3.fqdn.server,DMA-elastic4.fqdn.server</DBServer>
So each DMA has knowledge of each elastic node. I can see this in the SLSearch.txt logs and in the SLDBConnection.txt logs as well:
2021/07/14 15:21:07.653|SLDBConnection|SLSearch|INF|0|1|Connecting with: http://DMA-elastic1.fqdn.server:9200/ - http://DMA-elastic2.fqdn.server:9200/ - http://DMA-elastic3.fqdn.server:9200/ - http://DMA-elastic4.fqdn.server:9200/
We are assuming that all DMA will be communicating with DMA-elastic1.fqdn.server as it is the first item in the list. Is this the behavior or does each DMA randomly choose an Elastic node to send requests to?
When we bring down DMA-elastic1.fqdn.server, it is NOT observed in the log files (SLSearch.txt or SLDBConnection.txt) that the DMAs has detected that DMA-elastic1.fqdn.server is down and is trying to reach an alternative node in the Elastic cluster. We would like to validate that each DMA is in fact talking to each node in the elastic cluster when we randomly bring the elastic nodes offline. Where would be the best place to observe this on the DMA?
Hello,
This is a very intresting topic you bring forward.
First, the ip's or host names you mention in DB.xml are used to build the connection. If one host of those is down, the Elastic driver should pick up on this and just try the next one in the line. As soon as it finds one that is up, it's "connected".
However, once the connection is build, the Elastic driver is sniffing Elastic for all hosts. Once it fetched all hosts in the cluster, it basically does a round robin over all online hosts to send its queries. (more info at: Elastic Sniffing Pool Documentation)
So, the takeaway from this is: it doesn't matter how many ip's you fill in the db.xml, as long as one of them is online. The requests are being sent to random ip's (even ones that are not mentioned in db.xml).
I hope this answers your question,
Kind regards,
We use a driver, called the NEST driver. This driver allows us to choose on “how” we send requests to Elastic. We use the sniffing pool. This basically means that the driver will “sniff” out what hosts are in the elastic cluster and send requests to anyone of the online hosts in the Elastic cluster.
So, perhaps let’s clarify with an example, current situation:
Elastic Cluster containing node A, B and C.
DB.xml referencing A,B.
As soon as the connection was made to A or B, the driver will sniff that node C is online in the cluster and add it to a list of hosts. This lists is then being used to send requests to at random. So, your DataMiner will end up sending requests to node A,B and C. (you should view it as automatic load balancing)
Hi Thomas, this is really helpful with understanding how DMA interfaces with an Elastic cluster. One last follow-up question: In a DMA cluster, each DMA has its own db.xml file specifying Elastic nodes within the tag. Does each DMA form its own view of the Elastic cluster based upon the db.xml file or do all DMAs in the same cluster attempt to synchronize a common view of the Elastic cluster?
Hi Thomas, thanks for the insights in to how DMA in a cluster would interact with Elastic nodes in a cluster. I am able to follow your explanation up until the very end when you mentioned – “The requests are being sent to random ip’s (even ones that are not mentioned in db.xml).”. Can you please elaborate a little further on this point? I do not quite grasp why would DMA send requests to IPs that are not specified in db.xml tag?