Output feature store to S3

Question

I am trying to output the feature store toS3 bucket via a Sagemaker Notebook, however I get this error

```
`---------------------------------------------------------------------------
UnexpectedStatusException                 Traceback (most recent call last)
Cell In[22], line 1
----> 1 job_result = sess.wait_for_processing_job(processing_job_name)
      2 job_result

File /opt/conda/lib/python3.10/site-packages/sagemaker/session.py:4413, in Session.wait_for_processing_job(self, job, poll)
   4399 """Wait for an Amazon SageMaker Processing job to complete.
   4400 
   4401 Args:
   (...)
   4410     exceptions.UnexpectedStatusException: If the processing job fails.
   4411 """
   4412 desc = _wait_until(lambda: _processing_job_status(self.sagemaker_client, job), poll)
-> 4413 _check_job_status(job, desc, "ProcessingJobStatus")
   4414 return desc

File /opt/conda/lib/python3.10/site-packages/sagemaker/session.py:6986, in _check_job_status(job, desc, status_key_name)
   6980 if "CapacityError" in str(reason):
   6981     raise exceptions.CapacityError(
   6982         message=message,
   6983         allowed_statuses=["Completed", "Stopped"],
   6984         actual_status=status,
   6985     )
-> 6986 raise exceptions.UnexpectedStatusException(
   6987     message=message,
   6988     allowed_statuses=["Completed", "Stopped"],
   6989     actual_status=status,
   6990 )

UnexpectedStatusException: Error for Processing job data-wrangler-flow-processing-27-14-47-58-fcb5a019: Failed. Reason: ClientError: API error (404): pull access denied for 599662218115.dkr.ecr.eu-west-2.amazonaws.com/sagemaker-data-wrangler-container, repository does not exist or may require 'docker login': denied: User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/AmazonSageMaker-ExecutionRole-20240320T110821/SageMaker is not authorized to perform: ecr:BatchGetImage on resource: arn:aws:ecr:eu-west-2:599662218115:repository/sagemaker-data-wrangler-container because no resource-based policy allows the ecr:BatchGetImage action`
```

Answer

Hello,

I would recommend you to mask your account-id while posting in public forum.

Looking at the exception in the logs that you have shared , Can you please take a look at the role arn:aws:sts::xxxxxxxxx:assumed-role/AmazonSageMaker-ExecutionRole-20240320T110821/SageMaker has a policy that has **ecr:BatchGetImage** permission ?

Alternatively you can assign AmazonSageMakerFullAccess policy to the role to check if this resolves the issue and then write new least privilege policy based on the need from this.

Answer

To resolve this issue, you need to ensure that the IAM role associated with your SageMaker notebook instance has the necessary permissions to access the ECR repository.

Grant ECR permissions: Update the IAM role policy attached to your SageMaker notebook instance to include permissions for ecr:BatchGetImage on the specified ECR repository.

You can add the following statement to the IAM role policy:

```
{
    "Effect": "Allow",
    "Action": "ecr:BatchGetImage",
    "Resource": "arn:aws:ecr:eu-west-2:XXXXXXXX:repository/sagemaker-data-wrangler-container"
}
```
Ensure that the Resource ARN matches the ARN of your ECR repository.

Restart SageMaker notebook instance: After updating the IAM role policy, restart the SageMaker notebook instance to apply the changes.

Retry the operation: Once the SageMaker notebook instance is restarted and the IAM role has the necessary permissions, retry the operation to output the feature store to the S3 bucket.

If you continue to encounter issues, double-check the IAM role permissions and ensure that the ECR repository exists in the specified region.

Output feature store to S3

Relevant content