With the recommendation from Laurens of polling a large SNMP table 50 cells at a time within a QAction, we saw significant improvements to the time it takes to poll some tables we have (more than 40 thousand rows). However, we're still seeing that polling the tables is quite slow when compared to the device simulator tool or a MIB browser - a difference of 15 minutes with a tool, vs 21 minutes with DataMiner.
How can we still improve the polling speed of large SNMP tables?
If we reduce/increase the number of cells, would that change the amount of time it takes to poll the tables?
One of the reasons you could be experiencing a slower time to retrieve the same data could be the communication and data processing overhead variation of using different polling methods.
You could check if this is the case by seeing which polling method the external tool is using to perform the gets.
On this page (Link), you can see an overview of different methods of polling data, and in the comparison section, you see that the main disadvantage of the GetNext method (i.e. retrieve cell by cell) is that it is slow.
When communicating with devices, we always try to use the methods that retrieve the most data per request (i.e. retrieving multiple rows at once).
When this results in scenarios where the device cannot send all the data of a single request in time, we sometimes have to tweak slightly the amount of cells/rows we retrieve or even going with a different polling method.
So in your case, I would try to increase the number of cells to achieve a point where you have some better performance without having issues in retrieving the data.
I would suggest to use a tool like Wireshark to look into the difference between the get requests sent by your protocol and the other tool to check if you could further tweak the requests in your protocol.
Don’t go too high with the cell count. The content in the response should not be larger than a UDP packet (which SNMP is using). I had issues if the response is spread over multiple UDP packets. UDP is data that is not confirmed as opposite with TCP, so your packet(s) could be dropped by a router and SLSNMPManager also doesn’t seem to like having to reconstruct multiple UDP packets back together. Similar as the issue that Aldrick mentioned when trying to request the PreEq data, which is very large
Thanks for the quick feedback!
The tool I spoke of was the QADeviceSimulator tool (internal). Do you know the polling method is there? I believe you can select GetNext or GetBulk, and we used GetBulk in the tool. It does the default request amount there?
I know there is some internal DataMiner processing for the data, which we tried to workaround using QAction polling. I will look into testing with a higher cell count and look at the results.