I got this error in the upgrade when using the BrokerGateway.
error|Prerequisite VerifyNatsCluster.dll failed: Unable to create connection with endpoints nats://10.120.7.10:4222
Migrating back to the old NATs resolved the issue
It is one of the DMA's yes, but it looks like the old dll in the upgrade package, as I saw the same issue when the BPA on the system didn't update on our prod system, and all the test failed.
The cluster is talking fine on NATs though, using both new and old methods.
VerifyNatsCluster not only checks the primary communication paths but also ensures that all backup communication routes between agents are working.
This is important because, in some cases, an agent may be able to connect to its local NATS server without issues, yet still fail to reach a remote NATS server directly. By verifying these backup paths, we can detect and prevent communication problems that would otherwise only appear in cross-server scenarios.
That is probably the reason why the cluster is talking fine on NATs still. But if for some reason a local NATS node went down, there might be no access to the backup.
What does the error only occur when the BrokerGateway is used?
I thought the latest version of standard NATs created connections to all agents as well, so would have thought that same prerequisite would fail?
A connection to old NATS and BrokerGateway NATS is the same code.
So, yes. both versions can connect to all nats nodes of the cluster.
However, these connections are only made when connecting to the local nats node of the agent fails.
In case of the prerequisite, it forces a message to be sent between all possible pairs of nats nodes in the cluster. And 'skipping' the local node.
For example with 3 agents in a cluster the following combinations are tested:
Agent A => Agent B (and vice versa)
Agent A => Agent C (and vice versa)
Agent B => Agent C (and vice versa)
In the logline it just seems that the prerequiste itself cannot connect to that nats node in particular. So it didn't even get to the part of sending messages around.
In order to know the reason why the connection fails, more information will be needed from the logs.
Hi,
Does the IP 10.120.7.10 belong to an agent in the cluster?
If so, it would mean that there is an issue with the nats-server instance running on that agent or that the firewall prevents communication via port 4222.
Can you verify if this is the case?
Kind regards,
Pieter Lust