There is a project that requires development and integration of an urban data platform (UDP / Datalake) as a central data storage facility for a digital twin.
Does DataMiner support data lake as an option for storing the data gathered from the different devices?
Hi Petar - yes and no 😉
No, in the sense that DataMiner has its own fully integrated data storage solution (which is based on industry standard storage solutions) which are required to have a fully functional DataMiner System. So, you cannot replace this with another third-party data lake solution.
Yes, in the sense that you can easily offload data which has been collected / aggregated / processed / unified /... (depending on what you leverage DataMiner for) by DataMiner, into that third party data platform. There are various ways to get data from DataMiner into a third-party data storage, depending on the use cases and technical details.
Does that make sense? Feel free to circle back if you have further questions on the subject.
Hi Ben, thank you for your quick response
The requirement is to use a data lake as the primary data storage, capable of providing georeferenced data for visualization and analysis in various applications.
The data should be collected from various sensor systems or imported into the data lake, and it should be returned from the data lake for visualization. Clients should have the ability to visualize the data in different forms and create their own visualizations (e.g., using toolboxes or low-code platforms).
If we use third party platform to store the data collected by DataMiner, how challenging is it to establish and maintain constant communication with a third-party platform for storing and fetching data, and then using this data in DataMiner?
Hi Petar – I believe I may have misread your initial question then. This is not about integrating with an existing data lake. Instead, the requirement is to create a solution that creates a data lake, which can then be consulted for visualization. The data should be accessible for various tools.
And that’s essentially what you would be getting with a DataMiner System. A DataMiner System comes with industry standard big data storage solutions (either Cassandra & Elastic, or for STaaS that’s mainly Azure Cosmos DB) where it stores the data (and to some point also stored data in-memory, i.e. all most up to date values). This data can be used in DataMiner natively (alarming, visual overview graphics, trend / performance data), but also in third party applications (e.g. through various APIs, but for example third party tools like Grafana).
But if you have specific requirements for a specific project, I would maybe recommend that we jump on a call with a solution architect. That would probably be the best way to move this forward.
Hi Petar,
I see that this question has been inactive for some time. Do you still need help with this? If not, could you select the answer (using the ✓ icon)?