AWS EMR WAL creation error

Question

Hi

I am getting error while launching EMR with Hbase as S3Storage and WAL backup enabled .

Caused by: java.lang.RuntimeException: createWal failed for wal WALMetadata(WALWorkspace=testworkspace2, TablePrefix=, WALName=)

software.amazon.awssdk.services.emrwal.model.EmrwalException: User: arn:aws:sts:::assumed-role/AmazonEMR-InstanceProfile-.../i-... is not authorized to perform: EMRWAL:CreateWAL on resource: arn:aws:emrwal:...:workspace/testworkspace2/wal/...-because no identity-based policy allows the EMRWAL:CreateWAL action (Service: Emrwal, Status Code: 400, Request ID:....)
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
	at org.apache.hadoop.hbase.regionserver.wal.EMRWalClientWrappe

As per https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-wal-permissions.html
EMR instance profile must have  AWSServiceRoleForEMRWAL service-linked role to create WAL, and this role must be added by EMR at cluster launch automatically, My cluster is not getting this role added, I have enabled "iam:CreateServiceLinkedRole","iam:PutRolePolicy" for my instance profile , still AWSServiceRoleForEMRWAL is not being added.

What are the reasons of this?

Answer

For enabling EMR WAL below are the pre-requsites which are required as mentioned in  [1]:

-- Please add the below policies to EMR-EC2 instance profile role:

Policy 1 -

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "emrwal:DeleteWal",
                "emrwal:CreateWal",
                "emrwal:CreateWorkspace",
                "emrwal:AppendEdit",
                "emrwal:ReplayEdits",
                "emrwal:GetCurrentWalTime",
                "emrwal:CompleteWalFlush"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
```

Policy 2 -

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:PutRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/aws-service-role/emrwal.amazonaws.com*/AWSServiceRoleForEMRWAL*",
            "Condition": {
                "StringLike": {
                    "iam:AWSServiceName": [
                        "emrwal.amazonaws.com",
                        "elasticmapreduce.amazonaws.com"
                    ]
                }
            }
        }
    ]
}
```

-  Can you please attach the policy’s  and see how it goes.

References:

[1]: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase-wal-permissions.html

AWS EMR WAL creation error

Relevanter Inhalt