Hello All,
We have a solution that offers the ability to easily create many scheduler tasks that are expected to execute at the same time. This generally has worked ok in the past, but has never been attempted for hundreds of concurrent tasks.
Recently, ~900+ concurrent tasks were created and upon execution, they all failed with a LAST RUNTIME RESULT of "Incorrect function".
At exactly 6:07am in the Windows event viewer - we found the following error:
Which I believe indicates the DMA server became overwhelmed with the number of tasks being executed at one time.
Are there known limitations and/or best practices when it comes to the number of tasks that can be executed concurrently?
I was not able to determine expectations from Docs.
Thank you in advance!
Hi Thomas,
We did some testing a while back to figure out the limits of DataMiner scheduler and the underlying Windows task scheduler. Based on our testing these are the recommended limits:
- Max 100 concurrent tasks at the same time (per DMA)
- Max 3000 tasks in total (per DMA)
As you have noticed, the Windows task scheduler can become unreliable if there are too many tasks. The actual limit of windows task scheduler is system-dependent, these numbers are what we consider safe.
I have created a pull request to add these numbers to the docs as well.
Could you share some details about your use-case here? There might be a more performant/better way to achieve what you want.