EMR on EKS: Validation Error on creating managed endpoint for EMR on EKS

Question

I am getting a validation error, when I try to create a managed endpoint for EMR on EKS.

```
{
    "type": "JUPYTER_ENTERPRISE_GATEWAY",
    "state": "TERMINATED_WITH_ERRORS",
    "releaseLabel": "emr-6.9.0-latest",
    "stateDetails": "Failed to validate the network connectivity for the managed endpoint. Please check the failure reasons mentioned on the endpoint public documentation.",
    "failureReason": "VALIDATION_ERROR",
    "tags": {}
} 
```

Here is the command I am running.

```
aws emr-containers create-managed-endpoint --region us-west-1 \
--type JUPYTER_ENTERPRISE_GATEWAY \
--virtual-cluster-id $VIRTUAL_CLUSTER_ID \
--name emr-eks-endpoint \
--execution-role-arn $EMR_ROLE_ARN \
--release-label emr-6.9.0-latest \
--configuration-overrides '{
    "monitoringConfiguration": {
        "cloudWatchMonitoringConfiguration": {
            "logGroupName": "/test/emr",
            "logStreamNamePrefix": "test"
        }
    }
}'
```

Haven't found anything helpful on the logs, or on the documentation. I appreciate the help in advance.

Answer

A managed node group must be associated with an Amazon EKS cluster’s security group, which is usually the case if you created your cluster and managed node group using eksctl. You can verify this in the AWS console using the following steps.

1. Go to your cluster in the Amazon EKS console.
2. Go to the networking tab of your cluster and note down the cluster security group.
3. Go to the compute tab of your cluster and click on the managed node group name.
4. Under the Details tab of the managed node group, verify that the cluster security group that you noted previously is listed under Security groups.

If the managed node group is not attached to the Amazon EKS cluster security group, you need to attach the for-use-with-emr-containers-managed-endpoint-sg=ClusterName/NodeGroupName tag to the node group security group. Use the steps below to attach this tag.

1. Go to the Amazon EC2 console and click on security groups on the left navigation pane.
2. Select your managed node group’s security group by clicking the checkbox.
3. Under the Tags tab, add the tag for-use-with-emr-containers-managed-endpoint-sg=ClusterName/NodeGroupName using the Manage tags button.
Please refer this [document](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/managed-endpoint-parameters.html) to include the changes. In case the security group rules not allowed, then the endpoint might not be able to connect due to LoadBalancer unable to deploy traffic to the instances of the node group.

EMR on EKS: Validation Error on creating managed endpoint for EMR on EKS

Relevant content