Help me troubleshoot this problem.

Question

Hey, I connected to AWS MSK from pc using AWS CLI SSO Login. I made that cluster security group open to all TCP ports. Now I want to create a topic in the cluster here from my pc, using the cluster's *public* broker endpoint with this command. 
`.\bin\windows\kafka-topic.bat --create --topic my-topic --bootstrap-server b-1.democluster1.abc123.ac4.kafka.ap-south-1.amazonaws.com:9198 --partitions 2 --replication-factor 1 --command-config .\config\clients.properties`

It is giving error something like this.
```
Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler; for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler; could not be found.
        at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:747)
        at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:493)
        at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:486)
        at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:112)
        at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:145)
        at org.apache.kafka.clients.admin.AdminClientConfig.(AdminClientConfig.java:245)
        at org.apache.kafka.clients.admin.Admin.create(Admin.java:134)
        at kafka.admin.TopicCommand$TopicService$.createAdminClient(TopicCommand.scala:204)
        at kafka.admin.TopicCommand$TopicService$.apply(TopicCommand.scala:208)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:52)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
```

1. I have put *aws-msk-auth-iam-2.2.0-all.jar* in ./libs file only.
2. Here is my *clients.properties* code

```
sasl.mechanism=AWS_MSK_IAM
security.protocol=SASL_SSL
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler;
```

re:Post Agent · Answer

1. The error message suggests that the IAMClientCallbackHandler class could not be found. This is likely because the JAR file containing this class is not in the classpath. Even though you've placed the aws-msk-auth-iam-2.2.0-all.jar file in the ./libs directory, it may not be automatically included in the classpath.

2. To resolve this, you need to explicitly add the JAR file to the classpath when running the kafka-topics command. You can do this by setting the CLASSPATH environment variable before running the command. Try the following:

```
   set CLASSPATH=.\libs\aws-msk-auth-iam-2.2.0-all.jar
   .\bin\windows\kafka-topics.bat --create --topic my-topic --bootstrap-server b-1.democluster1.abc123.ac4.kafka.ap-south-1.amazonaws.com:9198 --partitions 2 --replication-factor 1 --command-config .\config\clients.properties
   ```

3. Make sure that the path to the JAR file is correct. If the libs folder is in a different location, adjust the path accordingly.

4. Your clients.properties file looks correct, but ensure there are no extra spaces or line breaks, especially at the end of each line.

5. Verify that you're using the correct version of the aws-msk-auth-iam JAR file that's compatible with your Kafka client version.

6. Double-check that your AWS CLI SSO login is still active and that your credentials are properly set up.

7. Ensure that the security group for your MSK cluster allows inbound traffic on port 9198 from your IP address, not just "all TCP ports" as this could be a security risk.

If these steps don't resolve the issue, you may need to check your Kafka client version compatibility with the MSK cluster version and ensure all necessary dependencies are correctly installed and configured.
      
**Sources**  
[Troubleshoot an IoT rule not sending messages in Amazon MSK | AWS re:Post](https://repost.aws/knowledge-center/msk-troubleshoot-iot-rule)  
[Troubleshoot your Amazon MSK cluster - Amazon Managed Streaming for Apache Kafka](https://docs.aws.amazon.com/msk/latest/developerguide/troubleshooting.html)

Help me troubleshoot this problem.

Relevant content