Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

What is the least impacting (safest) way to read a lot of DomInstances?

Solved1.06K views20th September 2023Automation CPU DOM Memory
2
Mieke Dryepondt [SLC] [DevOps Advocate]3.58K 20th July 2023 0 Comments

What is the advised method to use to avoid a massive load / memory usage when logic needs to retrieve DomInstances where the result could vary from 1 to 1 million (depending on the filter) ?

Option 1 (basic):

This will simply read All the DomInstances that matches the filter. So I'm expecting a big impact (timeout?) in case we are reading a lot of data.

public static List<DomInstance> GetAllDomInstanceByFieldValue(this DomHelper domHelper, FieldDescriptor fieldDescriptor, string value)

{

var filter = DomInstanceExposers.FieldValues.Field(fieldDescriptor.ID.Id.ToString()).Equal(value);

return domHelper.DomInstances.Read(filter);

}

Option 2 (paging):

This will get a set of DomInstances per go. (spreading the load somewhat)

public static List<DomInstance> GetAllDomInstanceByFieldValueByPages(this DomHelper domHelper, FieldDescriptor fieldDescriptor, string value)

{

var filter = DomInstanceExposers.FieldValues.Field(fieldDescriptor.ID.Id.ToString()).Equal(value);

var allDomInstances = new List<DomInstance>();

var pagingHelper = domHelper.DomInstances.PreparePaging(filter);

while (pagingHelper.MoveToNextPage())

{

var currentPage = pagingHelper.GetCurrentPage();

allDomInstances.AddRange(currentPage);

}

return allDomInstances;

}

How many items are retrieved via 1 page?

Option 3 (Skyline.DataMiner.Net.Tools):

I found this, but I don’t know what it does or when this should be used:

public static List<DomInstance> GetDomInstances(this DomHelper domHelper, List<Guid> instanceIds)

{

var instances = Tools.RetrieveBigOrFilter(

instanceIds,

id => DomInstanceExposers.Id.Equal(id),

filter => domHelper.DomInstances.Read(filter));

return instances;

}

Mieke Dryepondt [SLC] [DevOps Advocate] Selected answer as best 20th September 2023

2 Answers

  • Active
  • Voted
  • Newest
  • Oldest
5
Thomas Ghysbrecht [SLC] [DevOps Enabler]4.80K Posted 20th July 2023 0 Comments

Hi Mieke,

If retrieving a huge amount of DomInstances would be required, paging is the way to go. Keep in mind that the example you provided will load all these DomInstances into memory. If these could potentially be a million, you could push SLAutomation out of memory. In that case, it is recommended to get a page, do your actions, and then get the next page. That way you are only using the amount of memory that one 'chunk' needs.

Do note that this is not a definitive solution as it always depends on the use case, but the general rules to keep in mind are:
- Try to limit the size of the messages sent between server & client
- Try to keep the number of objects loaded into memory in mind

Mieke Dryepondt [SLC] [DevOps Advocate] Selected answer as best 20th September 2023
4
Matthias Declerck [SLC] [DevOps Advocate]2.75K Posted 20th July 2023 0 Comments

Hi Mieke,

Option 1 is the most common way to do it if you expect not too many instances.
Option 3 is the same as option 1, but instead of using a field value it is based on a list of specific DOM instances. The tools method is a converter from a list of GUID to a big OR filter (database query.
Option 2 is recommended if you want to handle instances in chunks.
By default, this has a size of 500 objects.
So if your result is 1300 instances, then you'll have 3 pages.
You can then create a sort of subscription system to handle these chunks, like for instance filling up a table or so. However, with your current example, it will not change in behaviour. because you iterate all pages in same thread and send it all back in 1 list.

Hope this will help you further.

Matthias Declerck [SLC] [DevOps Advocate] Edited answer 20th July 2023
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

Web Applications exception in Cube due to invalid certificate 0 Answers | 0 Votes
Redundancy Groups and Alarming – Duplicate Alarms 0 Answers | 0 Votes
Correlation Engine: “Test rule” doesn’t result in a hit, despite functional rule 1 Answer | 3 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (109) Correlation (69) Correlation rule (52) Cube (151) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (140) driver (65) DVE (56) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (152) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin