Amazon EFS Update

Scenario - From Network Fileshare/device

This is the scenario where the files are generated or updated to some file share/network drives.

EC2 instance

Sometimes, the the files may be generated to a network device that your VPC may have access to, like an on-premise file share. But unfortunately the only file storage type that Fargate can mount is EFS (as of this writing). In such a scenario (assuming that there isn't any mechanism by which the files can be uploaded to S3 bucket) the two methods discussed before wouldn't be sufficient.

One way is to use an EC2 instance, with its sole purpose is to mount the external drive as well as the EFS and copy/sync files from the external drive to EFS. The life cycle of the EC2 instance may be determined based on how often the external drives to EFS sync needs to happen. It can be an instance that is running all the time or something that comes up at scheduled intervals, does the sync and then disappears.

Obviously, this is not a 'serverless' approach as we are dealing with an EC2 instance. We will need to worry about the AMI, the instance's security posture, patching etc. etc. depending on the organization's policy.
On the positive side, it can be a cheaper option to achieve what we are trying to accomplish. Also, being an instance one could login there (Session Manager/SSH) and inspect the files if needed.

A minimal example for this could be find here in this github repository.

The options discussed so far are DYI and none of it will be of much help if we are dealing with huge amount of data and/or the files from the source are made available to Amazon FSx, object storage systems other than S3 etc. AWS provides a powerful managed solution in the form of AWS DataSync, which can be used to fulfill the scenarios that were covered so far, plus a lot more. In the next part of this series, we will explore AWS Datasync.

Updating files in Amazon EFS - Part 3

Comments

Updating Files in Amazon EFS

Updating files in Amazon EFS - Part 1

More from this blog

A Hard Learned Lesson on AWS DataSync

Updating files in Amazon EFS - Part 2

Updating files in Amazon EFS - Part 1

ECS Deployment using docker-compose

Scenario - From Network Fileshare/device

EC2 instance

Command Palette

Comments

Updating Files in Amazon EFS

Updating files in Amazon EFS - Part 1

More from this blog

Scenario - From Network Fileshare/device

EC2 instance