What is the most effective method for website monitoring and function testing? Currently, we employ Selenium and Puppeteer, transmitting the metrics to Prometheus. However, we are contemplating a shift to Dataminer for these tasks.
Regrettably, I haven't come across any information pertaining to this on community.dataminer.services. Alternatively, would it be optimal to execute this on a separate system and retrieve the results/metrics from Prometheus?
If anyone has insights on website testing with Dataminer or can direct me to relevant sources, I would greatly appreciate it.
Best Regards,
Chris
Both ways look possible to me:
- Instead of transmitting the metrics to Prometheus, you could transmit them to the Data API
- Or you create an GQI ad-hoc datasource that gets the data from Prometheus
Hi Christoph,
This is definitely an interesting question! As far as I know, this has not been done before.
My first question would be how would you like to use DataMiner in your testing pipeline? Would you like to store the test results using DataMiner, trigger certain tests from schedules, use the Dashboards to create visuals based on the test results stored in Prometheus, or... ?
Hi Michiel,
I try to remove as much of Prometheus as possible and include it directly in the Dataminer. In the meantime I have thought more about it and thanks to you and the input from Wim I have decided how the pipeline will look like and how I will implement it.
Dataminer triggers the script -> The script is sent to e.g. Selenium grid and executed there > The generated metrics are sent to the Data API of Dataminer.
Thanks for the input.
Greetings Christoph
Hi Christoph,
I’m glad to see you found inspiration on how to move forward.
Just out of professional curiosity as I’m also working on testing and pipelines quite a lot, are there any specific issues or difficulties with Prometheus that made you decide to move away from it?
Hello Michiel,
Thank you. Prometheus has not caused us any problems and works wonderfully. However, we have made the decision that all systems should ultimately be mapped in Dataminer, the more direct the better (this means skipping Prometheus).
For now, however, we will send the metrics to the Data API and in parallel to Prometheus. This means that all our systems will remain intact until we have everything at the same level.
Thank you Wim.
The missing part was the “Data API” and I will try it asap.
Regards
christoph