AWS MSK IAM - Authentication Failure Access Denied Spring Boot

0

I have a spring boot app deployed on AWS EKS POD and have provisioned AWS MSK with IAM authentication they both are under the same VPC and roles has been configured as well as in MSK inbound rules the port 9098 has also being added.

To test connectivity between EKS and MSK i did telnet with broker name and port 9098 it was successfully connected as well when my run spring boot app in eks pod it gives the below error:

org.springframework.kafka.KafkaException: Send failed;nested exception in org.apache.kafka.common.errors. SaslAuthenticationException: [63a192cc-599-43e-bfe8-bc880e50c2e1]: Access Denied


org.apache. kafka.clients.Networkclient: [Producer clientId=producer-1] Connection to node -3 b-3.xxxx.xxxx.amazonaws.com/10.7.2.1:9098) failed authentication due to: [63a192cc-599-43e-bfe8-bc880e50

My spring boot kafka config:

ssl.truststore.location=path to trust file
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

Created a role in IAM and assigned the below policies to it:

{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:DescribeTopicDynamicConfiguration",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:DeleteTopic",
                "kakfa-cluster:CreateTopic",
                "kakfa-cluster:AlterTopicDynamicConfiguration",
                "kakfa-cluster:AlterTopic",
            ],
            "Resource": [
                "arn:AWS_EKS_CLUSTER_ARN/*",
                "*"
            ]
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeCluster",
                    "kafka-cluster:DeleteGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:WriteDataIdempotently",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:WriteData",
                "kakfa-cluster:DescribeTransactionalId",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:AlterTransactionalId",
            ],
            "Resource":"*"
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": "kakfa-cluster":DescribeGroup,
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:ReadData",
                "kakfa-cluster:DescribeTopic",
            ],
            "Resource": "*"
            
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}

im using this dependencies in my spring app:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>sts</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>apache-client</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.13</artifactId>
    <version>3.0.1</version>
</dependency>

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
    <groupId>software.amazon.msk</groupId>
    <artifactId>aws-msk-iam-auth</artifactId>
    <version>1.0.0</version>
</dependency>

1 Resposta
0

Dear Customer,

I see that there is an issue with your IAM role permissions. You have given all the actions with your EKS Cluster as resource which is not the expected resource. You should be giving the MSK Cluster ARN, topic, group as explained below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
                "kafka-cluster:DescribeCluster"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:cluster/MyTestCluster/abcd1234-0123-abcd-5678-1234abcd-1"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:*Topic*",
                "kafka-cluster:WriteData",
                "kafka-cluster:ReadData"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:topic/MyTestCluster/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:group/MyTestCluster/*"
            ]
        }
    ]
}

Please refer to: [+] https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#create-iam-access-control-policies

AWS
ENGENHEIRO DE SUPORTE
respondido há 7 meses

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas