Sharing files between VPCs

Question

My customer wants to share the CI/CD artifacts that they generate in their build environment to be shared in DEV/UAT/PROD and other environments. They are looking at using EFS for this file share, but they don't like the need to do VPC peering in order to make the EFS available across VPCs. They also dont want to use S3 as they are looking for a traditional file system which can be mounted and used during the build process.

One option I could think of is to setup a EFS file system in every VPC and sync from the build environment to the target VPC using AWS DataSync. It would not be an automatic sync, so they will have have to call "StartTaskExecution" the source file system changes.

Are there any other solutions which they can consider?

Accepted Answer

Some options:

1. Use EFS with VPC Peering and/or Transit Gateway - I would explore why the customer doesn't want to do this, since it seems to make the most sense, is the easiest to manage, most cost-effective, (and can be tightly locked down with routing, security groups, NACLs, etc.)

2. Use File Storage Gateway in each VPC, connected to a single S3 bucket; have the build environment write to it, and have the others in "read-only" mode that would access the files from their File Storage Gateway.  Does what you want, but would incur additional costs (File Storage Gateway plus EC2 instance in each VPC).

3. Use S3 and mount it using something like S3FS.  Typically works for low volumes and change rates, but can have performance and reliability issues at scale.

Sharing files between VPCs

Relevant content