When running simple code from an automation to retrieve data from different elements. We are getting significant different results for the execution time, we are trying to identify what factors could be affecting this.
var iDms = engine.GetDms();
var element = iDms.GetElement(new DmsElementId(dmaId, elementId));var sourcesTable = element.GetTable(3000);
var sourcesData = sourcesTable.GetData();var destinationsTable = element.GetTable(4000);
var destinationsData = destinationsTable.GetData();watch.Stop();
engine.GenerateInformation("@@@@ Done getting the data. Time spent: " + watch.ElapsedMilliseconds + " number of inputs retrieved: " + sourcesData.Count + " Outputs: " + destinationsData.Count);
For example when running the code above on element A we get this result
@@@@ Done getting the data. Time spent: 3291 number of inputs retrieved: 2560 Outputs: 1536 (Script 'TestTemp')
For element B which is a smaller element and therefore in theory should take less time to get the data the opposite is happening.
@@@@ Done getting the data. Time spent: 20152 number of inputs retrieved: 1024 Outputs: 512 (Script 'TestTemp')
Both elements are on the same DMA which is part of a cluster. So going from 3 seconds to 20 that's almost 7x difference. Apart from the size of the data getting retrieved, what other factors could be affecting the execution time?
The issue was solved in a dataminer release. The issue was with one call the get external parameter or get table which was returning 10x the information behind the scene. This was solved in the SLNet message that was getting returned.
As you are using the DMS interface from the Class Library, the data is retrieved via SLNet. I suggest to also check the load on the SLNet process at the time you are executing the script as this can have an impact on the time it will take to execute these requests.
Do you need to get all the data from these tables? If not, you could also apply additional filters, which could have a big impact on performance.
Where the data actually resides can play a significant part … e.g. polling data from another DMA with a different local DB – with a cassandra cluster, I’d expect time to go down, but haven’t tried this – will tune to see wat comes from the community – the state of the DB, performance of the other server, SSD drives, it all comes into play from what I’ve seen so far – but 3 to 20s isn’t a big deal from my pov – as long as there is no timeout, I’d be fine if my query gets the results without stressing too much the DMAs- I understand GQI is the new preferred way as it should allow to structure the query in an efficient way – but I’m too new to it – so waiting some more experienced members in the dojo