Lambda Layer Write to EFS

Question

I'm attempting to use Step Functions to run a series of parallel `bcp -out` commands to fully export a SQLServer database into EFS.

I have successfully bundled the Microsoft `bcp` command-line utility as a lambda layer and i'm attempting to run this via a golang lambda using provided.al2023 runtime.  I can successfully execute the `bcp -v` command as seen in the Cloudwatch logs from the lambda.  Additionally, I've mounted an EFS filesystem to the lambda with proper permissions.  I'm able to successfully write out a file in `/mnt/efts/data/format/Table.format` albeit an empty file from the golang code.  Unfortunately, I can't execute the command from golang and have it write anything to EFS.

Ultimately, I'm trying to run two `bcp` commands which should output the results (first: the format file, second: the table exported data itself) to the EFS mounted access point.  However, when running the first command to output the format file `bcp dbo.Table format nul -c -f /mnt/efs/format/Database/Table.format -S {server} -P {password} -u` - I get the error: "Unable to open BCP host data-file".

Does a lambda layer inherit the same write permissions to EFS as the lambda it's running in?

re:Post Agent · Answer

Yes, a Lambda layer inherits the same write permissions to EFS as the Lambda function it's running in. The layer is essentially bundled with your Lambda function at runtime, so any code or binaries in the layer operate with the same permissions as the Lambda function itself.

The error "Unable to open BCP host data-file" suggests an issue with how the `bcp` command is interacting with the file system, not a permissions problem. Since you've confirmed you can write an empty file to the EFS mount point from your Go code, the Lambda function does have write access to the EFS file system.

Here are a few things to check:

1. Make sure the directory structure exists before trying to write the file. For example, ensure that `/mnt/efs/format/Database/` exists before trying to write `Table.format` to it.

2. Check the syntax of your `bcp` command. The error "Unable to open BCP host data-file" often occurs when the `bcp` command can't access the input or output file specified. In your command `bcp dbo.Table format nul -c -f /mnt/efs/format/Database/Table.format -S {server} -P {password} -u`, the `nul` parameter might be causing issues in the Linux environment.

3. Verify that the EFS mount is properly set up and accessible at runtime. You can add debug code to list the contents of the mount point and check its permissions.

4. Ensure that the user running the Lambda function (which is typically the Lambda service account) has the necessary permissions to write to the EFS file system. The initial permissions mode for an EFS mount allows read-write-execute permissions to the owner (root), and read-execute permissions to the group (root) and others.

5. Consider using the full path to the `bcp` executable in your command if you're not already doing so.

Since you've confirmed you can write an empty file to EFS from your Go code, the issue is more likely related to how you're executing the `bcp` command rather than a permissions issue with the Lambda layer.  
**Sources**  
[Map network file share on a Linux WorkSpace | AWS re:Post](https://repost.aws/knowledge-center/workspaces-linux-map-network-file-share)  
[Network File System (NFS) level users, groups, and permissions - Amazon Elastic File System](https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-nfs-permissions.html)

Lambda Layer Write to EFS

Add your answer

Relevant content