We are seeing a memory leak on SLProtocol and I think it is related to the amount of data that comes from a device.
The connection is a web socket connection and the element is receiving messages so quick that the messages are stacking up in SLProtocol's queue because they are not process as fast. I see many of the following message on the element log file:
021/10/25 16:37:32.938|SLProtocol - 23368 - TWLER101-1917-IPG03|17752|CProtocol::SmartIPReadThreadFunc|DBG|-1|Still 4000 messages to process.
What is the meaning of this log message?
Is there a way to see the items that are waiting to execute in the SLProtocol Queue?
Miguel,
Having a big amount of messages coming in, does not always mean they are added to the slprotocol queue. This depends on the logic in connector.
The queue only contains <Group> items, basically your "to-do's".
Processing incoming messages can be done without adding them to a "to-do".
Do you know for sure the items are queuing up in the slprotocol queue?
I don't know exactly what the log message means but my guess is that it is not the slprotocol queue but the list of incoming data.
To make sure that these can be handled as quick as possible, the logic in the connector should make sure to release the entrypoint of the connector as soon as possible. For example, by adding the incoming data to a buffer that is processed in batches.
Have a look at these articles about the internal flow for more details:
https://community.dataminer.services/documentation/data-ingest-control-plane-internal-flow-concept/
https://community.dataminer.services/documentation/data-ingest-control-plane-internal-flow-qactions/
Mieke, I am guessing that it has nothing to do with the slprotocol queue. Instead, the incoming messages are sent at a frequency of 1 second. The processing of them are happening on a QAction. This QAction is sometimes taking about 2-3 seconds to process/parse the data. The result- I believe -is an accumulation of messages on SLProtocol that are just waiting for the QAction to process.