connector is not able to connect to aws MSK

0

Hi team, I am working on case that using confluentinc-kafka-connect-s3-10.2.2.zip plugin to create connector in MSK connect. The connector will connect to AWS MSK and save data to S3. Both the connector and MSK within the same VPC, and same security group. The security group is configured with "If an Amazon MSK cluster is being used, then make sure to add a rule that allows inbound traffic from itself." However, I am still getting below error: [same error for other brokers as well]

[Worker-062e488d0b4551b47] [2022-11-12 00:05:28,340] INFO [AdminClient clientId=adminclient-1] Failed authentication with b-1.*.kafka.ap-southeast-2.amazonaws.com/INTERNAL_IP ([8e115136-72d6-48fb-a18b-6d9b8de34bb1]: Access denied) (org.apache.kafka.common.network.Selector:616) [Worker-062e488d0b4551b47] [2022-11-12 00:05:28,343] ERROR [AdminClient clientId=adminclient-1] Connection to node -3 (b-1.*.kafka.ap-southeast-2.amazonaws.com/INTERNAL_IP) failed authentication due to: [8e115136-72d6-48fb-a18b-6d9b8de34bb1]: Access denied (org.apache.kafka.clients.NetworkClient:771) [Worker-062e488d0b4551b47] [2022-11-12 00:05:28,345] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager:232) [Worker-062e488d0b4551b47] org.apache.kafka.common.errors.SaslAuthenticationException: [8e115136-72d6-48fb-a18b-6d9b8de34bb1]: Access denied [Worker-062e488d0b4551b47] [2022-11-12 00:05:28,362] INFO App info kafka.admin.client for adminclient-1 unregistered (org.apache.kafka.common.utils.AppInfoParser:83) [Worker-062e488d0b4551b47] [2022-11-12 00:05:28,365] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235) [Worker-062e488d0b4551b47] org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1668211558346, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s) [Worker-062e488d0b4551b47] Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata

By the way, the aws MSK is enabled with SCRAM and IAM authentication; Default MSK worker configuration is applied.

Shall I config username/password in somewhere of connector or worker? or why the error comes? Thanks, Linwei

2 Answers
1

As per reference[1], if the Amazon MSK cluster that you want to use with your connector is a cluster that uses IAM authentication, then you must add the following permissions policy to the connector's service execution role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:Connect",
                "kafka-cluster:DescribeCluster"
            ],
            "Resource": [
                "cluster-arn"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:ReadData",
                "kafka-cluster:DescribeTopic"
            ],
            "Resource": [
                "ARN of the topic that you want a sink connector to read from"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:WriteData",
                "kafka-cluster:DescribeTopic"
            ],
            "Resource": [
                "ARN of the topic that you want a source connector to write to"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:CreateTopic",
                "kafka-cluster:WriteData",
                "kafka-cluster:ReadData",
                "kafka-cluster:DescribeTopic"
            ],
            "Resource": [
                "arn:aws:kafka:region:account-id:topic/cluster-name/cluster-uuid/__amazon_msk_connect_*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": [
                "arn:aws:kafka:region:account-id:group/cluster-name/cluster-uuid/__amazon_msk_connect_*",
                "arn:aws:kafka:region:account-id:group/cluster-name/cluster-uuid/connect-*"
            ]
        }
    ]
}

I kindly request you to check if you have setup the IAM policy as mentioned above. If issue still persisits, then in order to dive deep into issue, we require details that are non-public information. Please open a support case with AWS using reference[2].

[1] https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-service-execution-role.html [2] https://console.aws.amazon.com/support/home#/case/create

profile pictureAWS
SUPPORT ENGINEER
AkashD
answered a year ago
profile picture
EXPERT
reviewed 14 hours ago
0

Thanks @AkashD, that's right, I missed that part. and all good now. Cheers

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions