How to bring customer S3 bucket to Pcluster3.8.0 FSX/Lustre

Question

I have a pre-existing S3 bucket-in-different-account (which we use daily for old 2.x PCluster fsx-lustre). We'd like to use that pre-existing S3 bucket and use it for new clusters using fsx-lustre in PCluster 3.8.0.

**WHAT-DOESN"T-WORK**
So far I've resorted to hand-jamming permissions on random pcluster3 roles in order to give the PCluster3 ui access to the customers S3 bucket. All that gets me is PCUI sends the CloudFormation w/params for cluster creation. But, that fails in CloudFormation with...
```
Amazon FSx is unable to create Service-Linked-Role to access the S3 bucket. 
Ensure the IAM role or user you are using has the required permissions. 
For more details, visit https://docs.aws.amazon.com/fsx/latest/LustreGuide/setting-up.html#fsx-adding-permissions-s3. 
(Service: AmazonFSx; Status Code: 400; Error Code: BadRequest; Request ID: eb6ce2e0-6c1f-4c6f-bb7c-d71cad7ea27c; Proxy: null)
```
and the Stack is reverted...

**WHAT DOES WORK**
If I abandon the customers S3 bucket (again in a different account), and I use the s3 bucket that pcluster3 creates as part of the UI thing `parallelcluster-REDACTED-v1-do-not-delete` I'm able to get a successfully created cluster with an fsx/ mount.  
HOWEVER, I'd like to avoid copying everything over from the customer's S3 bucket to this S3 bucket.

alfreds · Accepted Answer

I made some progress, at least I got past that error message and onto the next error.

I already had cross-acount permissions working as I stated in the comments to both initial answers so that wasn't my issue.

I spelled out in detail what I had to do in my bug https://github.com/aws/aws-parallelcluster/issues/6122

basically, you have to go heavy spelunking into finding which pcluster3 role is needing the service-linked-role and add a statement to the already full (10 max) policy role.

Unfortunately, you'll then get the next productivity killer 
```
Amazon FSx is unable to import objects from the linked data repository. Please file a ticket at https://console.aws.amazon.com/support/home#/. While filing your ticket, please include your file system ID and name of the linked data repository.
```
So now I'm onto that.

Osvaldo Marte · Answer

To integrate your pre-existing S3 bucket from a different account with PCluster 3.8.0 FSx/Lustre, you'll need to establish cross-account access and ensure the appropriate permissions are set up. Follow these steps:

1. **Set Up Cross-Account Access:**
   - Create a role in your account (let's call it Account A).
   - Create another role in the target account (Account B) with a trusted policy that allows access from Account A.
   - This will enable seamless communication between the two accounts.

2. **Grant Access between Accounts:**
   - Define a trusted policy in Account A to allow access from Account B.
   - Similarly, create a trusted policy in Account B to allow access from Account A.

Trusted Policy for Account A:
   ```json
   {
     "Version": "2012-10-17",
     "Statement": {
       "Effect": "Allow",
       "Action": "sts:AssumeRole",
       "Resource": "arn:aws:iam::PRODUCTION-ACCOUNT-ID-A:role/[ROLE_NAME_ACCOUNT_A]"
     }
   }
   ```

Trusted Policy for Account B:
   ```json
   {
     "Version": "2012-10-17",
     "Statement": {
       "Effect": "Allow",
       "Action": "sts:AssumeRole",
       "Resource": "arn:aws:iam::PRODUCTION-ACCOUNT-ID-B:role/[ROLE_NAME_ACCOUNT_A]"
     }
   }
   ```

3. **Configure Permissions for Amazon FSx:**
   - Follow the guidelines provided in the Amazon FSx for Lustre documentation to set up the necessary permissions for accessing S3 buckets. You can find detailed instructions [here](https://docs.aws.amazon.com/fsx/latest/LustreGuide/setting-up.html#fsx-adding-permissions-s3).

Additional resources:
- [Unable to validate access to an S3 bucket](https://docs.aws.amazon.com/fsx/latest/LustreGuide/s3-validation-error.html)

Wanyi Chen · Answer

Hello, it seems that you would like to set up fsx cross account S3 bucket access. That will required some additional settings for cross-account bucket access. Could you check if the following link helpful? 
https://repost.aws/questions/QUPA2hF7XSRamoN8sXgyovBw/how-to-access-bucket-from-another-account-to-fsx

How to bring customer S3 bucket to Pcluster3.8.0 FSX/Lustre

Relevant content