AWS MSK IAM - Authentication Failure Access Denied Spring Boot

0

I have a spring boot app deployed on AWS EKS POD and have provisioned AWS MSK with IAM authentication they both are under the same VPC and roles has been configured as well as in MSK inbound rules the port 9098 has also being added.

To test connectivity between EKS and MSK i did telnet with broker name and port 9098 it was successfully connected as well when my run spring boot app in eks pod it gives the below error:

org.springframework.kafka.KafkaException: Send failed;nested exception in org.apache.kafka.common.errors. SaslAuthenticationException: [63a192cc-599-43e-bfe8-bc880e50c2e1]: Access Denied


org.apache. kafka.clients.Networkclient: [Producer clientId=producer-1] Connection to node -3 b-3.xxxx.xxxx.amazonaws.com/10.7.2.1:9098) failed authentication due to: [63a192cc-599-43e-bfe8-bc880e50

My spring boot kafka config:

ssl.truststore.location=path to trust file
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

Created a role in IAM and assigned the below policies to it:

{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:DescribeTopicDynamicConfiguration",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:DeleteTopic",
                "kakfa-cluster:CreateTopic",
                "kakfa-cluster:AlterTopicDynamicConfiguration",
                "kakfa-cluster:AlterTopic",
            ],
            "Resource": [
                "arn:AWS_EKS_CLUSTER_ARN/*",
                "*"
            ]
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeCluster",
                    "kafka-cluster:DeleteGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:WriteDataIdempotently",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:WriteData",
                "kakfa-cluster:DescribeTransactionalId",
                "kakfa-cluster:DescribeTopic",
                "kakfa-cluster:AlterTransactionalId",
            ],
            "Resource":"*"
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": "kakfa-cluster":DescribeGroup,
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}
{
    "version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowMskAccessCluster",
            "Effect": "Allow",
            "Action": [
                "kafka:ListScramSecrets",
                "kafka:GetBootstrapBrokers",
                "kafka:DescribeCluster",
                "kafka-cluster:Connect",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN"
        },
        {
            "Sid": "AllowMskAccessTopic",
            "Effect": "Allow",
            "Action": [
                "kakfa-cluster:ReadData",
                "kakfa-cluster:DescribeTopic",
            ],
            "Resource": "*"
            
        },
        {
            "Sid": "AllowMskAccessGroup",
            "Effect": "Allow",
            "Action": [
                    "kafka-cluster:DescribeGroup",
                    "kafka-cluster:AlterGroup",
            ],
            "Resource": "AWS_EKS_CLUSTER_ARN/*"
        }
    ]
}

im using this dependencies in my spring app:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>sts</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>apache-client</artifactId>
    <version>2.16.13</version>
</dependency>
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.13</artifactId>
    <version>3.0.1</version>
</dependency>

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
    <groupId>software.amazon.msk</groupId>
    <artifactId>aws-msk-iam-auth</artifactId>
    <version>1.0.0</version>
</dependency>

rahul
질문됨 일 년 전817회 조회
1개 답변
0

Dear Customer,

I see that there is an issue with your IAM role permissions. You have given all the actions with your EKS Cluster as resource which is not the expected resource. You should be giving the MSK Cluster ARN, topic, group as explained below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:Connect",
                "kafka-cluster:AlterCluster",
                "kafka-cluster:DescribeCluster"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:cluster/MyTestCluster/abcd1234-0123-abcd-5678-1234abcd-1"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:*Topic*",
                "kafka-cluster:WriteData",
                "kafka-cluster:ReadData"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:topic/MyTestCluster/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": [
                "arn:aws:kafka:us-east-1:0123456789012:group/MyTestCluster/*"
            ]
        }
    ]
}

Please refer to: [+] https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#create-iam-access-control-policies

AWS
지원 엔지니어
답변함 7달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠