Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Empower Replay: Limited Edition
    • Tutorials
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • System Architecture
      • DataMiner Releases & Updates
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

How to achieve pagination for the GQI script in LCA for table?

261 views24th January 2025
1
Apurva Tandon [DevOps Advocate]1.61K 23rd January 2025 0 Comments

Hi I have code:

public GQIPage GetNextPage(GetNextPageInputArgs args)
{
_logger.Information(“Fetching next page”);
var rows = new List<GQIRow>();
List<GetParameterResponseMessage> paramInfo = new List<GetParameterResponseMessage>();
try
{
foreach (var service in serviceList)
{
var childInfos = service.Children?.ToList();
if (childInfos.Count > 0)
{
var reqParam = new List<DMSMessage>();
foreach (var childInfo in childInfos)
{
var paramsNeeded = childInfo.Parameters
.Where(x => _resources.Any(y => y.ParamterId == x.ParameterID))
.ToList();
foreach (var parameter in paramsNeeded)
{
var msg = new GetParameterMessage(childInfo.DataMinerID, childInfo.ElementID,
parameter.ParameterID, parameter.FilterValue, true);
reqParam.Add(msg);
}
}

try
{
var responseMsg = _dms.SendMessages(reqParam.ToArray());
var response = responseMsg.OfType<GetParameterResponseMessage>().ToList();
paramInfo = response.FindAll(x => !string.IsNullOrEmpty(x.Value.ToString()) && !x.Value.ToString().Equals(“EMPTY”, StringComparison.CurrentCultureIgnoreCase)).ToList();
}
catch (Exception e)
{
_logger.Error($”Error fetching parameters: {e.Message}”);
continue;
}

if (paramInfo.Any())
{
foreach (var childInfo in childInfos)
{
var cells = InitializeTableCells();
CreateElementRow(ref rows, childInfo, cells, paramInfo, service.Name);
}
}
}
}
return new GQIPage(rows.ToArray()) { HasNextPage = false };
}
catch (Exception e)
{
_logger.Error($”Error fetching elements: {e.Message}”);
return new GQIPage(rows.ToArray()) { HasNextPage = false };
}
}

In this code how can I achieve pagination as the GetNextPage is called every time as I have trigger attached to table, so how can pagination work to get data fast without hanging the LCA

Sebastiaan Dumoulein [SLC] [DevOps Enabler] Answered question 24th January 2025

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
0
Sebastiaan Dumoulein [SLC] [DevOps Enabler]12.78K Posted 24th January 2025 6 Comments

Hi Apurva,

In your current design, you’re first building the complete dataset before passing it to GQI. However, GQI then chunks this data into pages since the client typically only needs the first page (tables apply lazy loading).

The main issue is that constructing the entire dataset is time-consuming, even though you likely don’t need all of it at once. I recommend modifying the design to return data in pages directly to GQI. This way, if the client only requests the first page, only that portion of data from the ad hoc data source will be processed.

For your specific example, you should break the services list into manageable chunks and maintain a pivot in memory to track which services have been returned and which still need to be computed.

Ronald Gerard [SLC] [DevOps Advocate] Posted new comment 24th January 2025
Apurva Tandon [DevOps Advocate] commented 24th January 2025

Hi, Thanks,
Can we have some example, at some point they want to see whole data in the table, but thats fine if it loads in chunks, so not sure how to handle at GetNextPage…
I did try something but it only returned the first 10 or last 10 rows if I set page size to 10
Code:

public class PaginationState
{
public int LastProcessedServiceIndex { get; set; }
public int LastProcessedParameterIndex { get; set; }
}

private PaginationState _paginationState = new PaginationState();
private const int PageSize = 10;
public GQIPage GetNextPage(GetNextPageInputArgs args)
{
_logger.Information("Fetching next page");
var rows = new List<GQIRow>();
List<GetParameterResponseMessage> paramInfo = new List<GetParameterResponseMessage>();

int currentRowCount = 0; // To keep track of how many rows have been processed so far

try
{
_logger.Information($"TEST: {_paginationState.LastProcessedServiceIndex}");
// Iterate through services starting from the last processed service index
for (int serviceIndex = _paginationState.LastProcessedServiceIndex; serviceIndex < serviceList.Count; serviceIndex++)
{
var service = serviceList[serviceIndex];
var childInfos = service.Children?.ToList();
_paginationState.LastProcessedParameterIndex = 0; // Reset the parameter index
if (childInfos.Count > 0)
{
var reqParam = new List<DMSMessage>();
// Iterate through child services starting from the last processed parameter index
for (int childIndex = _paginationState.LastProcessedParameterIndex; childIndex < childInfos.Count; childIndex++)
{
_logger.Information($"TEST12: {_paginationState.LastProcessedParameterIndex}");
var childInfo = childInfos[childIndex];
var paramsNeeded = childInfo.Parameters
.Where(x => _resources.Any(y => y.ParamterId == x.ParameterID))
.ToList();

foreach (var parameter in paramsNeeded)
{
var msg = new GetParameterMessage(childInfo.DataMinerID, childInfo.ElementID,
parameter.ParameterID, parameter.FilterValue, true);
reqParam.Add(msg);
}

try
{
var responseMsg = _dms.SendMessages(reqParam.ToArray());
var response = responseMsg.OfType<GetParameterResponseMessage>().ToList();
paramInfo = response
.FindAll(x => !string.IsNullOrEmpty(x.Value.ToString()) && !x.Value.ToString().Equals("EMPTY", StringComparison.CurrentCultureIgnoreCase))
.ToList();
var cells = InitializeTableCells();
CreateElementRow(ref rows, childInfo, cells, paramInfo, service.Name);
currentRowCount++;
_logger.Information($"Count: {currentRowCount}");

// Stop if we've processed enough rows for the current page
if (currentRowCount >= PageSize)
{
break;
}
}
catch (Exception e)
{
_logger.Error($"Error fetching parameters: {e.Message}");
continue;
}

// Stop processing if we've reached the page size
if (currentRowCount >= PageSize)
{
_paginationState.LastProcessedParameterIndex = childIndex + 1; // Store where we left off
break;
}
}

// Stop processing if we've reached the page size
if (currentRowCount >= PageSize)
{
_paginationState.LastProcessedServiceIndex = serviceIndex + 1; // Store where we left off
break;
}
}

// Stop processing if we've reached the page size
if (currentRowCount >= PageSize)
{
break;
}
}

// Determine if there are more rows to process
bool hasNextPage = currentRowCount < rows.Count;

// Return the rows for the current page
_logger.Information(hasNextPage.ToString());
_logger.Information(currentRowCount.ToString() + " " + rows.Count.ToString());
return new GQIPage(rows.ToArray())
{
HasNextPage = hasNextPage
};
}
catch (Exception e)
{
_logger.Error($"Error fetching elements: {e.Message}");
return new GQIPage(rows.ToArray()) { HasNextPage = false };
}
}

Ronald Gerard [SLC] [DevOps Advocate] commented 24th January 2025

Assuming the call to CreateElementRow always adds a row to the rows list, the currentRowCount will always be equal to rows.Count, so the hasNextPage = currentRowCount < rows.Count will always be false and GQI will think there are no more rows after you return the first 10.

Apurva Tandon [DevOps Advocate] commented 24th January 2025

Hi Ronald,
Yes it does add an row, but I think when HasNextPage = true it does reitinalizes all the values in class again, now I tweaked the code to send rows per service wise , then also pagination fails
Hope the below value is not initialized again

public static int LastProcessedServiceIndex = 0;
public GQIPage GetNextPage(GetNextPageInputArgs args)
{
_logger.Information("Fetching next page");
var rows = new List<GQIRow>();
List<GetParameterResponseMessage> paramInfo = new List<GetParameterResponseMessage>();
try
{
_logger.Information("SERVICE COUNT" + serviceList.Count.ToString());
_logger.Information($"TEST SERVICE: {LastProcessedServiceIndex}");
var service = serviceList[LastProcessedServiceIndex];

var childInfos = service.Children?.ToList();
if (childInfos.Count > 0)
{
var reqParam = new List<DMSMessage>();
for (int childIndex = 0; childIndex < childInfos.Count; childIndex++)
{
var childInfo = childInfos[childIndex];
var paramsNeeded = childInfo.Parameters
.Where(x => _resources.Any(y => y.ParamterId == x.ParameterID))
.ToList();

foreach (var parameter in paramsNeeded)
{
var msg = new GetParameterMessage(childInfo.DataMinerID, childInfo.ElementID,
parameter.ParameterID, parameter.FilterValue, true);
reqParam.Add(msg);
}

try
{
var responseMsg = _dms.SendMessages(reqParam.ToArray());
var response = responseMsg.OfType<GetParameterResponseMessage>().ToList();
paramInfo = response
.FindAll(x => !string.IsNullOrEmpty(x.Value.ToString()) && !x.Value.ToString().Equals("EMPTY", StringComparison.CurrentCultureIgnoreCase))
.ToList();
var cells = InitializeTableCells();
CreateElementRow(ref rows, childInfo, cells, paramInfo, service.Name);

}
catch (Exception e)
{
_logger.Error($"Error fetching parameters: {e.Message}");
continue;
}
}
}
LastProcessedServiceIndex++;

hasNextPage = LastProcessedServiceIndex != serviceList.Count;
return new GQIPage(rows.ToArray())
{
HasNextPage = hasNextPage
};
}
catch (Exception e)
{
_logger.Error($"Error fetching elements: {e.Message}");
return new GQIPage(rows.ToArray()) { HasNextPage = false };
}
}

Ronald Gerard [SLC] [DevOps Advocate] commented 24th January 2025

Hi Apurva,
When HasNextPage is set to true, GQI does not reinitialize any values. The same object instance will be used to call GetNextPage again.

In this new code here however, since LastProcessedServiceIndex is static, that value is shared by separate, independent query executions and they will conflict and overwrite each other, giving an undesirable result.

Apurva Tandon [DevOps Advocate] commented 24th January 2025

Ok so for this piece
var service = serviceList[LastProcessedServiceIndex];

how can I handle this LastProcessedServiceIndex, so that it remains same for all query and not overwrite..I did remove static but still issue persits

and also,
on getNextPage call again when HasNextPage = true, will var rows = new List<GQIRow>();
be initialised again?

Show 1 more comments
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs