Hi Dojo,
I have been working on GQI Data Source to use in LCA, however it seems like components get stuck on loading when OnInit method takes a while. GQI is counting number of resources in the system and in cases where that number is small, the GQI works as intended and correct values are shown, however if that number is large, and operation is longer, it gets stuck on loading.
Since we use the same logic in one of our drivers and it works for faster operations I'm confident that issue is not caused by the code.
I've also noticed that the same operation on the same system takes longer for GQI then it does for the driver. Any reason why this might be the case?
Only conclusion I could think of so far is that GQI has a timeout, or maximum duration it can run for, before it moves on, but this is just an educated guess.
Can someone provide explanation of what is happening under the hood and how I might solve this?
Thank you,
Cheers
Hi Edib,
Currently there is only a timeout for the NATS communication between SLHelper (where GQI runs) and SLNet, which is 15 minutes by default I believe.
If that timeout would expire, you should get a clear error message for that in the UI.
Depending on the DataMiner version you're running, the logging and/or metrics found in C:\Skyline DataMiner\Logging\GQI can be helpful to debug this further. Potential errors and other important events are logged there.
You can temporarily enable "Debug" logging for more details (see: GQI logging docs).
Furthermore, in the /Metrics subfolder you can find out which requests where sent and how long they took if they finished.
From within your data source you can also add additional logging to investigate further (see: GQI extension logging docs).
As for why the same operation takes longer for GQI, that's hard to say for sure without knowing exactly which code you are executing or where you are measuring from.
There will likely be a bit more overhead in the communication via GQI (since it is fully generic after all), but that should not be so significant.
A possible explanation may be that multiple query sessions are active at the same time, and all trying to calculate the same thing and slowing each other down?
For example, if the LCA app page was refreshed a few times, the previous computation might still be active and impeding the flow.
Also note that if multiple components are using the same data source in a slightly different query, a new instance of your data source will also be created for each query.
For those scenarios, the best solution is often to cache the result in a static variable somewhere so multiple instance of your data source can use the same result without recalculating.
On a final note: it's usually not a good idea to do a lot of computation in the OnInit lifecycle method unless you have a good reason to do so.
The method is primarily designed to provide dependencies and do some quick initialization. It will also be triggered when just determining the query arguments and result columns.
Feel free to share any of your findings or code snippets if you need further assistance.
Hi Edib,
As the resources are all cached, the call ”
var resources = resourceHelper.GetResources(allResourcesFilter)” shouldn’t be an issue.
It would be useful to log how many times the query is executed and how long each execution takes when opening the application. Then, compare this with the time taken when the same query is executed in a QAction.
This might reveal if the query is being used multiple times, causing several requests that could delay the GQI overall.
Introducing a cache could be highly beneficial as Ronald mentioned.
Additionally, since RN39743, we have introduced the ability to fetch resources in a paginated manner, which could help in some cases.
Hi Andre,
The issue I’m facing with logging is that logging is done in batches, and since this line will never complete logging never happens, it could be that I just didn’t wait long enough but I did wait for ~10 minutes few times without logs appearing. I basically can’t use logging because of that. Also regarding multiple calls to get resources, I’ve tested with just one GQI and the results were the same.
The workaround I used is to get all resources for each pool, similar idea like one used in RN39743.
I’m now trying to make second query wait for the first one to be done, but I’m not having too much luck with it. I think I will go with just Thread.Sleep unless there is a reason not to do it that way or there is a better way.
Thanks
Hi Edib, logging does happen in batches because they are processed in a background thread, but this does not dependent on a specific amount of logs. So you should still be able to see logs that occur before the line where it gets stuck?
As for the question of waiting on a query, could you include a (pseudo)code snippet of what you’re trying to do exactly? Thread.Sleep most likely will not be the right solution.
Hi Ronald,
int timeout = 120;
while (PoolProvider.IsRunning && timeout– > 0)
{
Thread.Sleep(1000);
}
var pools = PoolProvider.GetResourcePools(resourceHelper);
PoolProvider.GetResourcePools(resourceHelper) is called from 2 different queries, and in order to avoid running them twice I include IsRunning to wait until the first query is done, at which point this call to get resource pools will use cached data and avoid call GetResources(which happens inside of GetResourcePools) all together.
Hi Ronald,
Thank you for your response. I’ve been able to identify the bottle neck but I don’t have an idea how to resolve it.
pools = new Dictionary();
// get all resource pools
var resourcePools = resourceHelper.GetResourcePools();
foreach (var rp in resourcePools)
{
pools.Add(rp.GUID, new Pool(rp.GUID, rp.Name));
}
// get all resources
var allResourcesFilter = new TRUEFilterElement();
var resources = resourceHelper.GetResources(allResourcesFilter);
foreach (var r in resources)
{
foreach (var pg in r.PoolGUIDs)
{
if (pools.TryGetValue(pg, out var pool))
{
var resource = new PoolResource(r.GUID, r.Name);
pool.Resources.Add(r.GUID, resource);
}
}
}
I get all pools and all resources and then expose total amount of resources as well as number of resources per pool in the GQI. Line at which query gets stuck is:
var resources = resourceHelper.GetResources(allResourcesFilter);
Note that it gets stuck only when number of resources is big. I’ve also moved calculation to the GetNextPage method of the GQI.
Any ideas why this might be happening?