Any ideas on what permission is blocking the AWS Transfer connector from downloading the file and saving it to the bucket ?

Question

Hi,
A lambda function in python using an AWS connector to download a file from a remote sftp server into a s3 bucket.
The response from the connector-id in the AWS CLI is (Got the connector-id from a print statement in the python code)

[cloudshell-user@ip-xx ~]$ aws transfer list-file-transfer-results --connector-id c-zzzzzzzzz --transfer-id 19f3a002-288d-4a46-8524-691ce71294e5
{
    "FileTransferResults": [
        {
            "FilePath": "test.csv",
            "StatusCode": "FAILED",
            "FailureCode": "WRITE_FILE_ERROR",
            "FailureMessage": "Error while writing file: Access denied. Check if your access role has necessary permissions"
        }
    ]
}

To test if the permissions are valid in the lambda function, the following code worked successfully to write a file to the S3 bucket.

# Define the bucket name and file path
    bucket_name = 'test-bucketname'
    file_path = 'inbound-files/testfile.txt'
    
    # Define the content of the file
    file_content = 'This is a test file for S3 write access.'
    
    # Upload the file to S3
    s3.put_object(Bucket=bucket_name, Key=file_path, Body=file_content)
	
Then looking at the AWS Transfer family connector, it has the following trust relationship and policies

Trust Relationship
```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "transfer.amazonaws.com",
                    "secretsmanager.amazonaws.com",
                    "s3.amazonaws.com",
                    "lambda.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
```
Policies
```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:GetRandomPassword",
                "secretsmanager:ListSecrets",
                "secretsmanager:BatchGetSecretValue",
                "secretsmanager:*"
            ],
            "Resource": "arn:aws:secretsmanager:ap-southeast-2:secret:secret:secret-secret-secret"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::test-bucketname/*"
            ]
        }
    ]
}
```
Any ideas on what is blocking the AWS Transfer connector from downloading the file and saving it to the bucket ?

Leo K · Accepted Answer

Note that the permissions you're granting are massively excessive. The `s3:*` permission, particularly when granted to the whole bucket, allows reconfiguring the bucket's permissions, destroying all its contents, sharing it with anyone in the world, or anything else imaginable with unlimited permissions.

With practically unlimited permissions, your transfers will certainly work, but so will any number of unintended actions that shouldn't happen.

For the IAM role used by the Transfer service, you should only allow the service principal `transfer.amazonaws.com` to assume the role, not the other listed service principals. The role's trust policy should look like this, with the account ID placeholder replaced by your account ID, to prevent other AWS users from configuring their Transfer services to assume your role and accessing your data:
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "transfer.amazonaws.com"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "000000000000"
        }
      }
    }
  ]
}
```

The permission policy attached to the role should look something like the following. This policy allows reading objects from and writing them to the bucket, as well as listing the objects in the bucket, but not reconfiguring the bucket's permissions, lifecycle policies, encryption or logging settings, or other potentially highly destructive actions.
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowBucketAccess",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::test-bucketname"
    },
    {
      "Sid": "AllowObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:GetObjectACL",
        "s3:PutObject",
        "s3:PutObjectACL"
      ],
      "Resource": "arn:aws:s3:::test-bucketname/*"
    },
    {
      "Sid": "GetConnectorSecretValue",
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "arn:aws:secretsmanager:ap-southeast-2:000000000000:secret:aws/transfer/SecretName-6RandomCharacters"
    }
  ]
}
```

re:Post Agent · Answer

Based on the information provided, there are a few potential reasons why the AWS Transfer connector is failing to download the file and save it to the S3 bucket:

1. S3 Bucket Policy: While the IAM role has permissions to access the S3 bucket, there might be a restrictive bucket policy on the S3 bucket itself that's preventing the Transfer connector from writing to it. Check the bucket policy to ensure it allows the Transfer service to write objects.

2. Encryption: If the S3 bucket is using server-side encryption with AWS KMS, the IAM role might be missing the necessary permissions to use the KMS key. You may need to add KMS permissions to the role policy.

3. Incomplete S3 Permissions: The current policy allows all S3 actions on objects within the bucket, but it doesn't explicitly allow actions on the bucket itself. You might want to add the bucket ARN without the "/*" to ensure full access to the bucket as well as its contents.

4. SFTP Server Connectivity: Ensure that the SFTP connector can successfully connect to the remote SFTP server. Check if the secret in Secrets Manager contains the correct credentials and connection details for the remote server.

5. File Permissions on Remote Server: Verify that the user account used to connect to the remote SFTP server has the necessary permissions to read the file you're trying to download.

To troubleshoot:

1. Review the CloudWatch logs for the Transfer connector for more detailed error messages.
2. Double-check the S3 bucket policy and ensure it's not restricting access.
3. If using KMS encryption, add the necessary KMS permissions to the IAM role.
4. Modify the S3 permission in the IAM role to include the bucket ARN without "/*".
5. Verify the connection details and credentials for the remote SFTP server in Secrets Manager.
6. Test the connection to the remote SFTP server manually to ensure the file is accessible.

If these steps don't resolve the issue, you may need to enable more detailed logging for the Transfer connector to pinpoint the exact cause of the failure.  
**Sources**  
[Six tips to improve the security of your AWS Transfer Family server | AWS Security Blog](https://aws.amazon.com/blogs/security/six-tips-to-improve-the-security-of-your-aws-transfer-family-server/)  
[Troubleshooting AWS Transfer Family - AWS Transfer Family](https://docs.aws.amazon.com/transfer/latest/userguide/troubleshooting.html)  
[Managing Transfer Family events using Amazon EventBridge - AWS Transfer Family](https://docs.aws.amazon.com/transfer/latest/userguide/eventbridge.html)  
[How to download file from others S3 bucket | AWS re:Post](https://repost.aws/questions/QUs5ia7jYbTAChIFL4pWxd6A/how-to-download-file-from-others-s3-bucket)

Any ideas on what permission is blocking the AWS Transfer connector from downloading the file and saving it to the bucket ?

Relevant content