EMR 6.15 shows IMDS Error on reading file from S3 bucket

Question

Hello,

I have upgraded the EMR from 6.14 to 6.15, and started seeing errors on the existing core node: 
`org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: IAMInstanceCredentialsProvider: Failed to load credentials from IMDS.`
and 
`FileDownloadException: Failed to download file path: s3a://abc.log.parquet, range: 0-123414, partition values: [empty row], isDataPresent: false, eTag: null`

On EC2 machines, IMDSv2 is optional. There is no such error on the primary node. I have also tried to get a response using the curl command for IMDSv1 and v2. I got responses for both of the commands.
There is a version upgrade for Hadoop (3.3.6) from older (3.3.3) for EMR-6.15.
Also, I'm not sure if AWS-SDK java would make any impact on this error.

I'm using EMR on EC2.

I'm looking to get help in debugging and solving the issue.

Answer

>org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: IAMInstanceCredentialsProvider: Failed to load credentials from IMDS

This issue was introduced in EMR 6.15 by s3a connector issue: https://issues.apache.org/jira/browse/HADOOP-18945
issues with aws sdk v2 migration on hadoop are tracked here : https://issues.apache.org/jira/browse/HADOOP-18886

> FileDownloadException: Failed to download file path: s3a://abc.log.parquet, range: 0-123414, partition values: [empty row], isDataPresent: false, eTag: null

could be the issue is with the parquet file, please validate this.

EMR 6.15 shows IMDS Error on reading file from S3 bucket

相關內容