MSK & KafkaJS - UNSUPPORTED_SASL_MECHANISM when using IAM authentication

0

I am receiving the error code UNSUPPORTED_SASL_MECHANISM when attempting to connect to my MSK cluster using KafkaJS. My cluster supports IAM and SASL/SCRAM authentication as shown in the below image:

MSK authentication options

Below is the code I am using to connect:

import {AclOperationTypes, AclPermissionTypes, AclResourceTypes, Kafka, ResourcePatternTypes} from "kafkajs";

const kafka = new Kafka({
    clientId: 'test',
    brokers: [
        `BROKER1`,
        `BROKER2`,
        `BROKER3`
    ],
    ssl: true,
    sasl: {
        mechanism: 'aws',
        authorizationIdentity: 'ROLE_ID',
        accessKeyId: 'ACCESS_KEY',
        secretAccessKey: 'SECRET_KEY',
    }
});

(async () => {
    const admin = kafka.admin()
    await admin.connect();
    console.log(await admin.describeAcls({
        resourceType: AclResourceTypes.ANY,
        resourcePatternType: ResourcePatternTypes.ANY,
        operation: AclOperationTypes.ALL,
        permissionType: AclPermissionTypes.ANY
    }))
})();

Below is the KafkaJS error output:

{"level":"ERROR","timestamp":"2023-03-09T14:17:16.414Z","logger":"kafkajs","message":"[Connection] Response SaslHandshake(key: 17, version: 1)","broker":"BROKER","clientId":"test","error":"The broker does not support the requested SASL mechanism","correlationId":1,"size":23}
{"level":"ERROR","timestamp":"2023-03-09T14:17:16.416Z","logger":"kafkajs","message":"[BrokerPool] The broker does not support the requested SASL mechanism","retryCount":0,"retryTime":346}
path\to\node_modules\kafkajs\src\protocol\error.js:581
  return new KafkaJSProtocolError(errorCodes.find(e => e.code === code) || unknownErrorCode(code))
         ^

KafkaJSProtocolError: The broker does not support the requested SASL mechanism
    at createErrorFromCode (path\to\node_modules\kafkajs\src\protocol\error.js:581:10)
  retriable: false,
  helpUrl: undefined,
  type: 'UNSUPPORTED_SASL_MECHANISM',
  code: 33
}

Below is the response received from the MSK cluster:

{ errorCode: 33, enabledMechanisms: [ 'AWS_MSK_IAM' ] }

The following line is also sent to CloudWatch Logs:

[2023-03-09 15:48:54,559] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Failed authentication with /MY_IPv4 (Unsupported SASL mechanism AWS) (org.apache.kafka.common.network.Selector)

I am also unable to run the same code above but using SASL/SCRAM instead of IAM authentication - I receive error code CLUSTER_AUTHORIZATION_FAILED.

The following is sent to CloudWatch Logs:

[2023-03-09 15:59:31,869] INFO Retrieving credential for user: USERNAME [INTERNAL]

I have also followed the steps outlined in this documentation but this has not had any effect.

1 Answer
0
Accepted Answer

It turns out that the AWS SASL mechanism that KafkaJS provides out of the box is not the one that AWS MSK uses!

I instead used this package https://github.com/jmaver-plume/kafkajs-msk-iam-authentication-mechanism to authenticate which worked immediately.

answered a year ago
profile picture
EXPERT
reviewed a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions