MSK & KafkaJS - UNSUPPORTED_SASL_MECHANISM when using IAM authentication

0

I am receiving the error code UNSUPPORTED_SASL_MECHANISM when attempting to connect to my MSK cluster using KafkaJS. My cluster supports IAM and SASL/SCRAM authentication as shown in the below image:

MSK authentication options

Below is the code I am using to connect:

import {AclOperationTypes, AclPermissionTypes, AclResourceTypes, Kafka, ResourcePatternTypes} from "kafkajs";

const kafka = new Kafka({
    clientId: 'test',
    brokers: [
        `BROKER1`,
        `BROKER2`,
        `BROKER3`
    ],
    ssl: true,
    sasl: {
        mechanism: 'aws',
        authorizationIdentity: 'ROLE_ID',
        accessKeyId: 'ACCESS_KEY',
        secretAccessKey: 'SECRET_KEY',
    }
});

(async () => {
    const admin = kafka.admin()
    await admin.connect();
    console.log(await admin.describeAcls({
        resourceType: AclResourceTypes.ANY,
        resourcePatternType: ResourcePatternTypes.ANY,
        operation: AclOperationTypes.ALL,
        permissionType: AclPermissionTypes.ANY
    }))
})();

Below is the KafkaJS error output:

{"level":"ERROR","timestamp":"2023-03-09T14:17:16.414Z","logger":"kafkajs","message":"[Connection] Response SaslHandshake(key: 17, version: 1)","broker":"BROKER","clientId":"test","error":"The broker does not support the requested SASL mechanism","correlationId":1,"size":23}
{"level":"ERROR","timestamp":"2023-03-09T14:17:16.416Z","logger":"kafkajs","message":"[BrokerPool] The broker does not support the requested SASL mechanism","retryCount":0,"retryTime":346}
path\to\node_modules\kafkajs\src\protocol\error.js:581
  return new KafkaJSProtocolError(errorCodes.find(e => e.code === code) || unknownErrorCode(code))
         ^

KafkaJSProtocolError: The broker does not support the requested SASL mechanism
    at createErrorFromCode (path\to\node_modules\kafkajs\src\protocol\error.js:581:10)
  retriable: false,
  helpUrl: undefined,
  type: 'UNSUPPORTED_SASL_MECHANISM',
  code: 33
}

Below is the response received from the MSK cluster:

{ errorCode: 33, enabledMechanisms: [ 'AWS_MSK_IAM' ] }

The following line is also sent to CloudWatch Logs:

[2023-03-09 15:48:54,559] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Failed authentication with /MY_IPv4 (Unsupported SASL mechanism AWS) (org.apache.kafka.common.network.Selector)

I am also unable to run the same code above but using SASL/SCRAM instead of IAM authentication - I receive error code CLUSTER_AUTHORIZATION_FAILED.

The following is sent to CloudWatch Logs:

[2023-03-09 15:59:31,869] INFO Retrieving credential for user: USERNAME [INTERNAL]

I have also followed the steps outlined in this documentation but this has not had any effect.

1 Antwort
0
Akzeptierte Antwort

It turns out that the AWS SASL mechanism that KafkaJS provides out of the box is not the one that AWS MSK uses!

I instead used this package https://github.com/jmaver-plume/kafkajs-msk-iam-authentication-mechanism to authenticate which worked immediately.

beantwortet vor einem Jahr
profile picture
EXPERTE
überprüft vor einem Monat

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen