AWS MSK IAM - Authentication Failure Access Denied Spring Boot

0

I have a spring boot app deployed on AWS EKS POD and have provisioned AWS MSK with IAM authentication they both are under the same VPC and roles has been configured as well as in MSK inbound rules the port 9098 has also being added.

To test connectivity between EKS and MSK i did telnet with broker name and port 9098 it was successfully connected as well when my run spring boot app in eks pod it gives the below error:

org.springframework.kafka.KafkaException: Send failed;nested exception in org.apache.kafka.common.errors. SaslAuthenticationException: [63a192cc-599-43e-bfe8-bc880e50c2e1]: Access Denied


org.apache. kafka.clients.Networkclient: [Producer clientId=producer-1] Connection to node -3 b-3.xxxx.xxxx.amazonaws.com/10.7.2.1:9098) failed authentication due to: [63a192cc-599-43e-bfe8-bc880e50

My spring boot kafka config:

ssl.truststore.location=path to trust file
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

Created a role in IAM and assigned the below policies to it:

{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:DescribeTopicDynamicConfiguration",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:DeleteTopic",
                "kakfa-cluster:CreateTopic",
                "kakfa-cluster:AlterTopicDynamicConfiguration",
                "kakfa-cluster:AlterTopic",
            ],
            "Resource": [
                "arn:AWS_EKS_CLUSTER_ARN/*",
                "*"
            ]
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeCluster",
                    "kafka-cluster:DeleteGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:WriteDataIdempotently",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:WriteData",
                "kakfa-cluster:DescribeTransactionalId",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:AlterTransactionalId",
            ],
            "Resource":"*"
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": "kakfa-cluster":DescribeGroup,
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:ReadData",
                "kakfa-cluster:DescribeTopic",
            ],
            "Resource": "*"
            
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}

im using this dependencies in my spring app:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>sts</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>apache-client</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.13</artifactId>
    <version>3.0.1</version>
</dependency>

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
    <groupId>software.amazon.msk</groupId>
    <artifactId>aws-msk-iam-auth</artifactId>
    <version>1.0.0</version>
</dependency>

rahul
質問済み 1年前816ビュー
1回答
0

Dear Customer,

I see that there is an issue with your IAM role permissions. You have given all the actions with your EKS Cluster as resource which is not the expected resource. You should be giving the MSK Cluster ARN, topic, group as explained below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
                "kafka-cluster:DescribeCluster"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:cluster/MyTestCluster/abcd1234-0123-abcd-5678-1234abcd-1"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:*Topic*",
                "kafka-cluster:WriteData",
                "kafka-cluster:ReadData"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:topic/MyTestCluster/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:group/MyTestCluster/*"
            ]
        }
    ]
}

Please refer to: [+] https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#create-iam-access-control-policies

AWS
サポートエンジニア
回答済み 7ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ