MSK & KafkaJS - UNSUPPORTED_SASL_MECHANISM when using IAM authentication

0

I am receiving the error code UNSUPPORTED_SASL_MECHANISM when attempting to connect to my MSK cluster using KafkaJS. My cluster supports IAM and SASL/SCRAM authentication as shown in the below image:

MSK authentication options

Below is the code I am using to connect:

import {AclOperationTypes, AclPermissionTypes, AclResourceTypes, Kafka, ResourcePatternTypes} from "kafkajs";

const kafka = new Kafka({
    clientId: 'test',
    brokers: [
        `BROKER1`,
        `BROKER2`,
        `BROKER3`
    ],
    ssl: true,
    sasl: {
        mechanism: 'aws',
        authorizationIdentity: 'ROLE_ID',
        accessKeyId: 'ACCESS_KEY',
        secretAccessKey: 'SECRET_KEY',
    }
});

(async () => {
    const admin = kafka.admin()
    await admin.connect();
    console.log(await admin.describeAcls({
        resourceType: AclResourceTypes.ANY,
        resourcePatternType: ResourcePatternTypes.ANY,
        operation: AclOperationTypes.ALL,
        permissionType: AclPermissionTypes.ANY
    }))
})();

Below is the KafkaJS error output:

{"level":"ERROR","timestamp":"2023-03-09T14:17:16.414Z","logger":"kafkajs","message":"[Connection] Response SaslHandshake(key: 17, version: 1)","broker":"BROKER","clientId":"test","error":"The broker does not support the requested SASL mechanism","correlationId":1,"size":23}
{"level":"ERROR","timestamp":"2023-03-09T14:17:16.416Z","logger":"kafkajs","message":"[BrokerPool] The broker does not support the requested SASL mechanism","retryCount":0,"retryTime":346}
path\to\node_modules\kafkajs\src\protocol\error.js:581
  return new KafkaJSProtocolError(errorCodes.find(e => e.code === code) || unknownErrorCode(code))
         ^

KafkaJSProtocolError: The broker does not support the requested SASL mechanism
    at createErrorFromCode (path\to\node_modules\kafkajs\src\protocol\error.js:581:10)
  retriable: false,
  helpUrl: undefined,
  type: 'UNSUPPORTED_SASL_MECHANISM',
  code: 33
}

Below is the response received from the MSK cluster:

{ errorCode: 33, enabledMechanisms: [ 'AWS_MSK_IAM' ] }

The following line is also sent to CloudWatch Logs:

[2023-03-09 15:48:54,559] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Failed authentication with /MY_IPv4 (Unsupported SASL mechanism AWS) (org.apache.kafka.common.network.Selector)

I am also unable to run the same code above but using SASL/SCRAM instead of IAM authentication - I receive error code CLUSTER_AUTHORIZATION_FAILED.

The following is sent to CloudWatch Logs:

[2023-03-09 15:59:31,869] INFO Retrieving credential for user: USERNAME [INTERNAL]

I have also followed the steps outlined in this documentation but this has not had any effect.

1 Respuesta
0
Respuesta aceptada

It turns out that the AWS SASL mechanism that KafkaJS provides out of the box is not the one that AWS MSK uses!

I instead used this package https://github.com/jmaver-plume/kafkajs-msk-iam-authentication-mechanism to authenticate which worked immediately.

respondido hace un año
profile picture
EXPERTO
revisado hace un mes

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas