What is Rabbit MQ and can it be integrated with DataMiner?
It's a message broker system. Yes it can be integrated with DataMiner, but one of the key things here is to be always specific about what we are talking. Because this is something that can be looked at from different angles. In DataMiner you can build a driver to interface with Rabbit MQ for example, but that's not really a solution yet.
Because the question is what the objective of the integration is:
Could be that people want to send data from within DataMiner across Rabbit MQ to other software applications connected to it.
Or it could be that somebody asks about Rabbit MQ, but in fact is asking about monitoring a third party system that can be interfaced with through Rabbit MQ.
Or somebody could have a high dependency on a Rabbit MQ installation, because he has a lot of operational systems that communicate with one another across it, and he could be interested to monitor the Rabbit MQ system itself.
So it is always very important to be aware of the exact objectives. This also applies if people talk about "can you integrate with Elastic". Is the purpose monitoring an Elastic cluster? Or is this about retrieving data from the Elastic cluster that was put in there by a third party system (so in fact in that case it is about monitoring the third party application, and the data to do that happens to be in that Elastic Cluster).
Whenever the underlying case is really about managing a third party application (e.g. via Rabbit MQ, via data pulled from an Elastic Cluster), the design has to be carefully reviewed. Because it can be tricky to do a proper integration with one generic Rabbit MQ or generic Elastic driver, after all the data from one third party system to another, albeit the fact that they are both on Rabbit MQ or on Elastic, can be completely different (in terms of structure, type of data, etc.). So while they share the same method to get to the data, they might well be completely different and separate DataMiner drivers and development efforts. The question is always how much is common and how much is specific.
Indeed Mieke. First step is always to define the objective, or one could be wasting a lot of time. The most tricky one is really when people talk about ‘can you integrate with Rabbit MQ or with Elastic” for example, when they are really talking about integrating another third party product, for which the data needs to be accessed through Rabbit MQ or by digging into the data in an Elastic Cluster.
Because ultimately there is a choice, to either try to make a generic driver that can somehow pull in data from those sources, irrespective of the underlying third party product. So the question is then, can one driver be used to monitor ANY third party system that connects to Rabbit MQ, or ANY third party system that dumps data into Elastic. And this really needs to be judged on case by case. It might be possible to make something generic, but it is almost inevitable that you will be making compromises in terms of the quality of the integration (as compared to making individual drivers / integrations for each individual third party product, even if they both deliver their data through Rabbit MQ or through an Elastic cluster).
Of course, if the volume of integrations that need to be done through a specific intermediate system, then it might be worth looking at how we can take the generic part of interacting with that intermediate system, and make that common for all drivers that we need. In other words, is it possible to separate the details of interfacing with Rabbit MQ in a way that later on multiple drivers can leverage it to extract data from third party systems via Rabbit MQ. This is more or less comparable to the fact that so many systems use SNMP, and DataMiner having SNMP natively on board, so that when you develop a driver for a product where you need to communicate via SNMP, you do not have to worry too much about the details of that protocol.
But again, this is not a black and white thing, and all cases need to carefully reviewed to come to the most sensible direction to take…
RabbitMQ is a general purpose messaging solution, often used to allow web servers to respond to requests quickly instead of being forced to perform resource-heavy procedures while the user waits for the result.
[1]
They provide a Java Client API Guide [2] which can be used to develop a DataMiner driver. There is also AMQP (their proprietary binary messaging protocol for microservices and enterprise messaging) documentation available for download on their official website [4].
Thank you Ben for your answer. So we should always consider the bigger context / need and integrate accordingly.