We are seeing a memory leak on SLProtocol and I think it is related to the amount of data that comes from a device.
The connection is a web socket connection and the element is receiving messages so quick that the messages are stacking up in SLProtocol’s queue because they are not process as fast. I see many of the following message on the element log file:
021/10/25 16:37:32.938|SLProtocol – 23368 – TWLER101-1917-IPG03|17752|CProtocol::SmartIPReadThreadFunc|DBG|-1|Still 4000 messages to process.
What is the meaning of this log message?
Is there a way to see the items that are waiting to execute in the SLProtocol Queue?
Hi Miguel,
The SmartIPReadThread is a thread running in SLProtocol used to process smart-serial responses. These responses are processed sequentially and the next one will only be processed when the previous one has been fully handled.
Are you triggering a QAction to process/parse these responses? You might want to look into how quickly responses are coming in compared to the time it takes to process them.
If the rate is too high, then a solution would be better to buffer the responses and process/offload them in bulk every x seconds (using a timer for example). This will make sure that your QAction finishes as soon as possible everytime a new request comes in. You could even do the processing in a different thread then.
Miguel,
Having a big amount of messages coming in, does not always mean they are added to the slprotocol queue. This depends on the logic in connector.
The queue only contains <Group> items, basically your “to-do’s”.
Processing incoming messages can be done without adding them to a “to-do”.
Do you know for sure the items are queuing up in the slprotocol queue?
I don’t know exactly what the log message means but my guess is that it is not the slprotocol queue but the list of incoming data.
To make sure that these can be handled as quick as possible, the logic in the connector should make sure to release the entrypoint of the connector as soon as possible. For example, by adding the incoming data to a buffer that is processed in batches.
Have a look at these articles about the internal flow for more details:
https://community.dataminer.services/documentation/data-ingest-control-plane-internal-flow-concept/
https://community.dataminer.services/documentation/data-ingest-control-plane-internal-flow-qactions/
Mieke, I am guessing that it has nothing to do with the slprotocol queue. Instead, the incoming messages are sent at a frequency of 1 second. The processing of them are happening on a QAction. This QAction is sometimes taking about 2-3 seconds to process/parse the data. The result- I believe -is an accumulation of messages on SLProtocol that are just waiting for the QAction to process.
Joey, I am parsing using a QAction. I used the Pending Calls on test tool and noticed that the QAction was sometimes running for 2-3 seconds. The frequency in which the device is sending data was set to 1 second. Changing the frequency to 4 seconds fixed the issue. I will advice the user about the method you recommended to avoid issues later on. Thanks!