EMR_DefaultRole has insufficient EC2 permissions

I was having the same errors and I followed Almas advice. However, the CloudTrail logs were not that helpful. What I found was a few instances of a InvalidRequestException, which turned out to be an ListYarnApplicationsPrivate event. The error message was NoSuchElementFound, and it gave me the cluster ID. The cluster ID did exist, though!

I did further debuging by copying the deprecated policy, AmazonElasticMapReduceRole, into a new policy. And then one by one deleting the Actions. E.g. deleting all the ec2:** actions, then all the iam:** actions, etc. This debugging led me to the real problem: iam:PassRole.

In the new policy, AmazonEMRServicePolicy_v2, we see these two statements toward the bottom of the code:

    {  
        "Sid": "PassRoleForAutoScaling",  
        "Effect": "Allow",  
        "Action": "iam:PassRole",  
        "Resource": "arn:aws:iam::*:role/EMR_AutoScaling_DefaultRole",  
        "Condition": {  
            "StringLike": {  
                "iam:PassedToService": "application-autoscaling.amazonaws.com*"  
            }  
        }  
    },  
    {  
        "Sid": "PassRoleForEC2",  
        "Effect": "Allow",  
        "Action": "iam:PassRole",  
        "Resource": "arn:aws:iam::*:role/EMR_EC2_DefaultRole",  
        "Condition": {  
            "StringLike": {  
                "iam:PassedToService": "ec2.amazonaws.com*"  
            }  
        }  
    }  

If you are like me, when I went to upgrade to the newer EMR access policies, I made a new EC2 role called EC2DefaultRoleforEMR. Because my name doesn't match the name in the AmazonEMRServicePolicy_v2 policy, my cluster was not getting the iam:PassRole permissions it needed.

In classic AWS fashion, this is mentioned in passing in the documentation: "The v2 policies incorporate new iam:PassRole security configurations,..." They fail to mention that everything will break if you come up with your own names for the EC2 role.

https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-roles.html

There are a lot of ways of solving this, but for me the approach that made the most sense was to make a policy called IAMPassRoleforEMR And make the contents something like:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PassRoleForAutoScaling",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": [
"arn:aws:iam::<acct_number>:role/EMR_AutoScaling_DefaultRole",
"arn:aws:iam::<acct_number>:role/AWSServiceRoleForEMRCleanup",
"arn:aws:iam::<acct_number>:role/EC2DefaultRoleforEMR",
"arn:aws:iam::<acct_number>:role/<your_role_names_here>",
"arn:aws:iam::<acct_number>:role/<etc>"
],
"Condition": {
"StringLike": {
"iam:PassedToService": "application-autoscaling.amazonaws.com"
}
}
},
{
"Sid": "PassRoleForEC2",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": [
"arn:aws:iam::<acct_number>:role/EMR_AutoScaling_DefaultRole",
"arn:aws:iam::<acct_number>:role/AWSServiceRoleForEMRCleanup",
"arn:aws:iam::<acct_number>:role/EC2DefaultRoleforEMR",
"arn:aws:iam::<acct_number>:role/<your_role_names_here>",
"arn:aws:iam::<acct_number>:role/<etc>"
],
"Condition": {
"StringLike": {
"iam:PassedToService": "ec2.amazonaws.com"
}
}
}
]
}

Hi Emmanuel Silva,

Hope you are doing well.
I understand you are facing issues related EMR_DefaultRole permissions while launching an EMR cluster and need guidance regarding the same.

To troubleshoot the issue further from your end, you can check API calls which were blocked by insufficient permissions by checking the AWS Cloudtrail Events.

1.Navigate to the Cloudtrail console: https://console.aws.amazon.com/cloudtrail/
2.Click on Event History tab
3.Then use Filter as "Event Source" and in Time range select the timestamp during cluster launch.
4.From the buttons on Right side, click on the Gear Icon, which is for Show/Hide columns and select the Error Code column check box.

Once all the above is done, go through the list of events and expand the one which has an ErrorCode like AccessDenied or any other exception.
Once you know which API call is being denied, you can then investigate further regarding the same.

Revert back on this thread for more assistance/guidance.

I hope this helps.

-Almas

create cluster with tag: for-use-with-amazon-emr-managed-policies=true. I believe it will propagate to EC2 instances allocated.