Dear Dojo Community,
I hope this message finds you well.
We are currently exploring a use case that involves offloading specific files from our DataMiner agent backups to AWS cloud storage. The primary goals of this initiative include enhancing our disaster recovery capabilities, establishing a data-driven archive, and implementing version control for the backed-up files.
In our current setup, we perform full backups for every DataMiner agent. However, for this project, we are particularly focused on offloading the backup files from three specific agents to AWS S3.
Attached, you will find a diagram illustrating our intended architecture. We are keen to understand if it is feasible to configure these agents to transfer the selected backup files directly to the AWS S3 bucket using network paths.
Has anyone in the community undertaken a similar project involving DataMiner and AWS S3? If so, I would greatly appreciate any insights, experiences, or best practices you could share.
Additionally, I would be grateful for your thoughts on the reliability and practicality of this approach.
Thank you in advance for your assistance. I look forward to your valuable feedback.
Hi Samson,
It seems that Amazon AWS S3 don't support natively UNC (Universal Naming Convention) paths (network paths). This means that you will not be able to upload directly DataMiner backups to AWS S3. However, there is a connector that can help you with this requirement: Amazon AWS S3 File Upload
Checking the documentation, this connector was made specifically to upload backups to AWS S3:
The purpose of this connector is to be able to upload files (specifically DataMiner backup files) towards the Amazon S3 cloud service using a DataMiner element.
Hope it helps.
Hi Samson,
Similar to AWS S3, AWS EFS don’t support natively UNC. Currently we don’t have a connector that supports uploading files to AWS EFS. Furthermore, Investigating a bit further, it seems that integrating AWS EFS in DataMiner (for backup purposes) will be challenging. AWS EFS is designed to be accessed via NFS protocol, which is primarily used in Linux OS. You could use either a third-party client to allow you the connection to AWS EFS, or use AWS Storage Gateway.
Thanks so much for this again Miguel!
Hello Miguel,
Thank you so much for your valuable suggestion.
We had another discussion today regarding this, and I wanted to explore an alternative. Could we consider using Amazon EFS instead of S3? We’re leaning towards EFS due to its advantages in Real-Time File Access & Delivery, Data Management, Compliance, Encryption (both at rest and in transit), and Latency Avoidance, among other factors.
Would EFS be a viable option for our needs?
Thanks again for your help!