NAS & NATS services: tolerated latency in Distributed clusters?

Solved1.14K viewsadl2099 NAS NATS sync
2
3 Comments

Hi Dojo,
Are there any known latency ranges that must be satisfied for NAS & NATS when DMA servers are on a global network?

What are the options and how can these affect synchronizations across the DMS?
Is stand-alone NATS node config faster than a Clustered NATS?

Which config can be lighter on SLNet.exe?

E.g. Let’s say I’m evaluating a DMS cluster consisting of a DMA1 in EU and a DMA2 in the US:
when I create a new “view” on DMA1, how long am I expected to wait before I can see the same view (or element) when connecting to DMA2?
What are the options to minimize the cluster synchronization in a scenario like this, involving intercontinental link delay?

Thanks

Alberto De Luca [DevOps Enabler] Edited comment 22nd June 2024
Alberto De Luca [DevOps Enabler] commented

PS: If/When running NAS/NATS without the automatic cluster settings, so in “stand-alone” mode, are there any changes to NATs port required or will the ports stay the same? (IPC TCP 4222, 6222, 8222)

https://community.dataminer.services/securing-dataminer-part-2-firewall/?hilite=NATS
Marieke Goethals [SLC] [DevOps Catalyst] commented

Hi Alberto,
I see that this question has been inactive for some time. Do you still need assistance with this? If not, could you select the answer to indicate that the question is resolved?

1 Answer

1
368 3 Comments

Hello Alberto,

I’m assuming the stand-alone NATS node config you’re mentioning is forcing the NATS configuration in a DMA cluster to not be clustered.
This is not a supported configuration and was only temporarily used to quickly get customer DMAs up and running again, after which a proper configuration has to be set up.
A stand-alone NATS configuration cannot communicate with other nodes of the DMA which makes many features not function correctly.
Hence this type of configuration should never be put in production.

So a clustered config is required. We don’t however have tests to determine a specific cutoff latency at which point NATS will stop functioning reliably.
We are however in the process of implementing a new project which will decouple the NATS management from DataMiner, which will make it easier to update, maintain and provide new features.
One of those features is support for super-cluster and leafnodes, which are things provided by NATS to optimize the connectivity in a geographically distributed cluster.

We currently have no ETA for the release of this new project since there are many hurdles yet to overcome.

I hope this answers all your questions

Alberto De Luca [DevOps Enabler] Selected answer as best 22nd June 2024
Alberto De Luca [DevOps Enabler] commented

Thanks for the thorough feedback, Laurens

With reference to the “emergency” scenario where NATS is no longer clustered,
is there any comms flow that is addressed via SLNet?

Even if not supported, I’d like to dive further in the details as an emergency work-around was applied in our environment, so I’m capturing the actions to revert this to the standard expected config.
Laurens Vergote [SLC] [DevOps Advocate] commented

Most of the communications still happen via existing channels (COM/.net remoting/grpc). Only some functionalities use NATS as communication backbone. However, some of those functionalities are essential to the normal operation of DataMiner.
In the future we want to put more communication on NATS when it makes sense to use NATS (broadcasting, eventing, …), but for targeted communication use cases, grpc is the better option.
Alberto De Luca [DevOps Enabler] commented

Thanks again for the feedback, Lauren – much appreciated

I’ll keep this open just a few days more, in case there’s anyone else in the community who’s got some more info in terms of tolerated latency over WAN & international links.

It’s good to know that “super-cluster” and “leaf nodes” will become available at some point and that ad interim NATs is a required piece of the backbone communications in nominal cluster conditions.