Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Updates & Insights
  • Questions
  • Learning
    • E-learning Courses
    • Empower Replay: Limited Edition
    • Tutorials
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
      • About the DevOps Program
      • DataMiner DevOps Support
  • Downloads
  • More
    • DataMiner Releases & Updates
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

Reduce number of HTTP table rows processed simultaneously in multi-threaded timer

Solved1.14K views18th October 2023HTTP Multi-Threaded Timer Multithreaded Timer threadPool
3
Chian Hao Aw [SLC] [DevOps Advocate]650 18th July 2023 0 Comments

I have a multi-threaded timer that is processing about 100+ table rows per second. This leads to a lot of API calls every 24 hours which places a lot of stress on the device communicating with DataMiner. After checking the documentation it seems like either decreasing the threadpool size or adding the polling rate (pollingRate:15,3,3) could help to reduce the number of requests being sent to the device over time.

the each parameter also affects how many rows are polled at a time it seems, but based on my calculations of the table rows in the driver vs the time period of 2 minutes, it should be processing 1.1 rows per second, which is not what the device is receiving.

Which method would be more effective? Other than longer overhead are there any other drawbacks that I may not account for when changing these parameters? Any other possible methods that could work better?

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 18th October 2023

2 Answers

  • Active
  • Voted
  • Newest
  • Oldest
2
Laurens Moutton [SLC] [DevOps Enabler]8.78K Posted 18th July 2023 2 Comments

How many rows are there in table 1000?

The "each" defines that a row should be polled every 2 minutes. If it cannot reach this rate it will start adding the rows to the waiting queue and then poll it as fast as possible to get rid of the waiting queue. However, a row can only be present once in the waiting queue, so I would not expect it to be executed multiple times within the 2 minutes interval. If the thread pool is decreased it's then not getting enough resources and rows are added to the waiting queue, better to decrease the "each" to have less polling.
What are the other parameters 6201->6206 indicating as value?
What do you mean with calls every 24 hours? Is the multi threaded timer being stopped/started in between?
Are timestamps added when a row gets polled? If so, what do you see when it is changing when focussing on one row? Is this changing around every 2 minutes or faster?

Laurens Moutton [SLC] [DevOps Enabler] Posted new comment 18th October 2023
Chian Hao Aw [SLC] [DevOps Advocate] commented 1st August 2023

Hi Laurens, apologies for the late response.

Table 1000 has about 155 rows in it. I added a pollingRate: 200,1,1 based on the PollingRateCalculator tool and wanted to share some of the results.

The max duration of polling the table is around 2 seconds, with an average duration of 1.1 seconds. The driver also uses a maximum of 63/100 threads available to make API calls.
My main focus was to spread out the API calls over time as there were reports that the device was being overwhelmed with too many calls, as many as 130/second which is about 480,000 in 24 hours. It’s been running for about a week now with no issues, but I’m wondering if there is anything left to improve performance.

Laurens Moutton [SLC] [DevOps Enabler] commented 18th October 2023

Hi, there’s not more that can be done to improve performance. Regards,

You are viewing 1 out of 2 answers, click here to view all answers.
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin