Hi,
In my page, there is a table component associated to a GQI query "Get object manager instances" for displayling data.
There is in database thousand instances for the DOM definition i chose and the number is set to increase considerably.
When i refresh the page, the loading time is reasonable (3-4 seconds).
I added a Column Custom Operator to the query. And now the loading time is excessive long (30 seconds).
If i 've understood the doc correctly, if i do not have filter, it is not possible to optimize custom operators, thousand instances will be scanned? Optimizing your custom operator | DataMiner Docs
How can i improve performance?
Do i need to create a custom GQI ad hoc source with paging?
public GQIPage GetNextPage(GetNextPageInputArgs args)
Do you have an example of usage?
Thanks
Kévin
Hi Kévin,
A custom operator is indeed a great way to extend, transform, and manipulate data. A custom operator will, for example, do something with each row (in your case, probably adding a value to the added column).
Unless some expensive operations are needed to generate that value, the impact of a custom operator on performance should not be significant.
However, there is one important sidenote: With GQI and the DOM Instances data source in this case, we try to fetch the data as efficiently as possible. This means that if a user adds a filter operator in the query or uses the filter option in the table header, we try to forward that filter as close to the database as possible. But if there is a custom operator in your query between the data source and the filter operator, we can't know if we are allowed to forward the filtering to the database. Because of that, we need to go into post-filtering mode, which means that GQI will do the filtering in memory, which will be less efficient.
To address this, we introduced the feature described in the following tutorial: https://docs.dataminer.services/user-guide/Advanced_Modules/Dashboards_and_Low_Code_Apps/GQI/Extensions/Custom_Operator/Tutorials/Custom_Operator_Tutorial.html
Through this mechanism, you can indicate that filter operators might be forwarded if applicable.
Does the above mentioned reason explain why the performance has decreased significantly? If so, I would suggest to apply the optimization as shown in the tutorial.
Feel free to let me know if anything is unclear, I can further assist you.
Best regards, Ward
Hi Ward,
Thanks for your feedback. Not sure to understand, because as you can see in the following screenshot (GQI query), there is no filter by default when the page is opened. There is a sorting operation to have the latest elements at the top.
Regards
Kévin
Hi Kévin,
The sort operator might be causing the performance decrease, similar to what we discussed regarding the filter operator. Typically, GQI tries to apply sorting as close to the database as possible. However, with a custom operator, it can’t do this automatically.
To resolve this, you’ll need to modify your custom operator to forward the sort operation to the database. This is the same optimization principle used for the filter operator.
Instead of the ‘OptimizeForFilter’ method you can use the ‘OptimizeForSort’ method. (https://docs.dataminer.services/user-guide/Advanced_Modules/Dashboards_and_Low_Code_Apps/GQI/Extensions/Custom_Operator/Tutorials/Custom_Operator_Tutorial.html)
Let me know if you have any questions!
Best regards,
Ward
EDIT: If you would apply the custom operator at the end of your query (after the sort) the performance of the query itself should be better already. But, when a user clicks the table header to apply sorting, behind the scenes an additional sort operator is added and performance might be worse again.
Hello,
the point is we need to display data coming from several type of objects linked together by relationship.
In a traditionnal SQL database like Postgres or Oracle, we would create an SQL view and map it to a specific entity and we could benefits of all the capabilities you mention.
Question is, is it possible to create views or something similar in Dom model ?
Hi Julien, sorry for the (extremely) late reply. It is, afaik, not possible to create views within the DOM model. With your GQI query you are in fact creating a view on your data layer. Did performing above suggested performance improvements help you? Best regards, Ward
I see that this question has been inactive for some time. Do you still need help with this? If not, could you select the most relevant answer (this can also be your own answer)?