Cannot get a VPC-peered EFS accesspoint to mount

OK, 100% the issue is that the EFS mount helper doesn't know the right region. When I update efs-utils.conf to hardcode the region to ca-central-1, the mount works perfectly. So now the question is how can I override the region in the mount code? Perusing the mount.efs script, it doesn't seem like that's possible.

Couple of things you should look at:

1: Since it is cross-region VPCs scenario, I would ask you to check and make sure if you are to resolve and connect EFS with ID over NFS port. As we stated in our docs, you must need to have EFS with mount target ID added into your client /etc/hosts file where you are trying to mount. https://aws.amazon.com/premiumsupport/knowledge-center/access-efs-across-accounts/ https://docs.amazonaws.cn/en_us/efs/latest/ug/efs-different-vpc.html

2: You can change the AZ in your mount options matching with that mount target IP address (even with region also should work).

retry after these two and should work. The NFS type mount you have used is using EFS IP address not the EFS ID, so there it will not look for DNS resolution with EFS ID to IP address.

1. The articles there assume that the mountpoint is in the same AZ as the EC2 instance. This isn't the case for us, as the mountpoint is in ca-central-1a, but the EC2 instance is in us-east-1a. Nevertheless, I have tried following those instructions multiple times and still no luck. The volume is mounted in ca-central-1a with an IP address of 10.1.137.225. When I run sudo mount -t efs -o tls fs-02a76f6147dba6d53:/ efs I get the following error

Failed to resolve "fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com". The file system mount target ip address cannot be found, please pass mount target ip address via mount options.

So as you can see, the EFS mount helper is trying to look up the FS based on the region that the instance is in. Nevertheless, I then update /etc/hosts to add an entry 10.1.137.225 fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com so that the helper knows where to go for that hostname. When I do that, the connection simply times out. Again though, we KNOW that this instance has a route to the mountpoint, because we can mount it with the command I listed in my first post.

2. When I change the mount command to sudo mount -t efs -o tls,az=ca-central-1a fs-02a76f6147dba6d53:/ efs, I get a

Failed to resolve "ca-central-1a.fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com". The file system mount target ip address cannot be found, please pass mount target ip address via mount options.

again, and as expected, when I update the hosts file to use this new hostname that the utility is looking for, the connection still times out.

Thank you for your help.

Can you update the default region in the efs-utils.conf file which should be under "/etc/amazon/efs/" path and then retry: Refer: https://docs.aws.amazon.com/efs/latest/ug/mount-different-region.html

I am trying to avoid doing this, because we also have EFS volumes from us-east-1 mounted on this same machine.

In that case, can you make sure you are adding right entry to /etc/hosts file ? The reason it is unable to resolve is looks like you have added the below entry where the EFS FQDN is with us-east-1 region instead of ca-central-1 region.. 10.1.137.225 fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com

Please retry after correcting that.

The reason I had that in the hosts file is because when I run the mount command, it complains that it cannot resolve the domain. I updated it to us-east-1 simply for troubleshooting. See below:

Hosts file: 10.1.137.225 fs-02a76f6147dba6d53.efs.ca-central-1.amazonaws.com

Command: sudo mount -t efs -o tls,iam fs-02a76f6147dba6d53:/ efs

Output: Failed to resolve "fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com". The file system mount target ip address cannot be found, please pass mount target ip address via mount options. The file system fs-02a76f6147dba6d53 is not found

And when I pass the target IP via the mounttargetip option as instructed, it simply times out.

Modify the /etc/hosts file entry as shown below:

10.1.137.225 fs-02a76f6147dba6d53.efs.ca-central-1.amazonaws.com fs-02a76f6147dba6d53

Then use one of below command to test the mount:

sudo mount -t efs -o tls,iam fs-02a76f6147dba6d53:/ efs

sudo mount -t efs -o tls,iam fs-02a76f6147dba6d53.efs.ca-central-1.amazonaws.com:/ efs

Modified as instructed, and again the error is:

Failed to resolve "fs-02a76f6147dba6d53.efs.us-east-1.amazonaws.com". The file system mount target ip address cannot be found, please pass mount target ip address via mount options. The file system fs-02a76f6147dba6d53 is not found

Because the mount command, seeing just the fs ID, is assuming that the mount is in the same region as the EC2 instance.

As an update to this, I enabled debug logging for the mount helper, and I can see that the request that botocore is issuing is going to the wrong region, which is why it can't find the filesystem

2022-08-04 17:49:21 UTC - DEBUG - Starting new HTTPS connection (1): elasticfilesystem.us-east-1.amazonaws.com:443
2022-08-04 17:49:21 UTC - DEBUG - https://elasticfilesystem.us-east-1.amazonaws.com:443 "GET /2015-02-01/mount-targets?FileSystemId=fs-02a76f6147dba6d53 HTTP/1.1" 404 97
2022-08-04 17:49:21 UTC - DEBUG - Response headers: {'x-amzn-RequestId': 'e3df4821-378e-4d12-8b02-6438b0d5e5e5', 'x-amzn-ErrorType': 'FileSystemNotFound:', 'Content-Type': 'application/json', 'Content-Length': '97', 'Date': 'Thu, 04 Aug 2022 17:49:20 GMT'}

I know the EC2 instance can retrieve metadata about the EFS volumes, because a aws efs describe-mount-targets --file-system-id fs-02a76f6147dba6d53 --region ca-central-1 works perfectly fine. I just need to somehow get the EFS mount helper to use the correct region.

my solution was to edit /etc/amazon/efs/efs-utils.conf and set region= and then use empty parameter az=

example

mount -t efs -o tls,az= fs-02a76f6147dba6d53:/ efs