- Newest
- Most votes
- Most comments
Hi, IAM permissions looks correctly. Instance profile which is used to spin EMR cluster instance has to be able to interact KMS key which will be used to encrypt EBS storage. You stated also that deployment was done in subnet with IGW. Can you verify if indeed IGW is in route table for that subnet? What I would suggest to check is to make sure that VPC is properly configured. KMS service must be accessible from subnet(s) where cluster nodes are being created. Preferred way one can achieve that is by creating VPC Endpoint for KMS service (https://docs.aws.amazon.com/kms/latest/developerguide/kms-vpc-endpoint.html#vpce-create-endpoint). This way will make sure traffic will not traverse via public internet but stay inside AWS network.
Hello,
I see you also indicated how you setup the EMR security configuration. if no insights on Ctrail, on your S3 log directory go to the prefix /cluster_ID/node/<master_node_ID>/setup-devices/ and refer to the log DiskEncryptor.log.gz for insights.
If you still not seeing any insights on WHY, please indicate will also test the same using your above steps
Hi, Thanks for your answers.
Finally, after many days I found the solution: DELETE aws:SourceArn and aws:SourceAccount CONDITIONS from trust relationships for Instance Profile.
In my case when this conditions are in trust relationships lunching flow ends with "local disk encryption error - Internal error", logs are not publishing on S3 and there's no info in CloudTrail.
EMR_EC2_DefaultRole:
Trust relationships:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole",
/*START DELETE */
"Condition": {
"StringEquals": {
"aws:SourceAccount": "<account-id>"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:elasticmapreduce:<my-region>:<my-account-id>:*"
}
}
/*END DELETE */
}
]
}
I don't know why in AWS Documentation they provide this condition in Trust relationships: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-role.html
Relevant content
- asked 2 years ago
- asked 2 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
You are mixing EMR Service Role and EC2 Instance Profile. In documentation you are referring to it is clearly stated that conditional block is for Service Role not Instance Profile. EMR EC2 Instance Profile creation is described here https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-role-for-ec2.html.