How do I resolve Amazon ECR image permissions issues or job issues that occur when I use custom images on Amazon EMR Serverless?

2 minute read
0

I want to resolve Amazon Elastic Container Registry (Amazon ECR) permissions issues or job issues that occur when I use custom images on Amazon EMR Serverless.

Resolution

Amazon ECR image permissions issues

If you start an Amazon EMR Serverless application and your application fails, then you receive an error message that's similar to the following one:

"Application testing (example-application-id) failed to start. EMR Serverless service principal is not authorized to perform: ECR:DescribeImages on resource."

To resolve this issue, complete the following steps:

  1. Open the Amazon ECR console.
  2. In the navigation pane, choose Repositories.
  3. Select a repository that has a custom image.
  4. On the menu, choose Permissions.
  5. Choose Edit Json Policy, and then enter the following permissions:
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Emr Serverless Custom Image Support",
          "Effect": "Allow",
          "Principal": {
            "Service": "emr-serverless.amazonaws.com"
          },
          "Action": [
            "ecr:BatchGetImage",
            "ecr:DescribeImages",
            "ecr:GetDownloadUrlForLayer"
          ],
          "Condition": {
            "StringEquals": {
              "aws:SourceArn": "arn:aws:emr-serverless:example-region:example-account-id:/applications/example-application-id"
            }
          }
        }
      ]
    }
    Note: Replace example-region with your AWS Region, example-account-id with your account ID, and example-application-id with your application ID.

Job issues

If your job fails when you run a custom image, then check the complete logs in the configured logging destination. If logs aren't available, then check the following configurations:

  • The base image for the custom image has the same type (Spark or Hive), Amazon EMR Serverless version, and architecture (x86_64/arm_64).
  • The custom image didn't change Amazon EMR to include jars or binary files.
  • The custom image environment variables, such as JAVA_HOME or SPARK_HOME, aren't changed.

Related information

Customizing an EMR Serverless image

AWS OFFICIAL
AWS OFFICIALUpdated 2 months ago