Hi,
I have an MSK cluster with SASL/SCRAM enabled and created appropriate Super User for cluster with Secret to manage further users in cluster.
If I create new users for example with Sarama client using this superuser as admin client, they appear in zookeeper config and kafka-configs, but connection for such users is failed with - Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512.
Can such internal Kafka users have access to MSK cluster with some additional configuration or every further user in MSK with SASL/SCRAM should have separate "Amazon_MSK_" prefixed secret and managed via KMS?
Error points to failed authentication. I think the user hasn't been created properly, or a client application doesn't define properties correctly. Please add more details, make sure the secret you are using is listed under MSK -https://docs.aws.amazon.com/cli/latest/reference/kafka/list-scram-secrets.html . Follow the documentation to properly define a new user for MSK: https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html
Let me clarify, I used only Kafka API and created user (No MSK secrets). In vanilla Kafka it works (all users are stored in ZK config), but in AWS such users fail authentication.
Can I use AWS admin user to create other users via default Kafka API? Or such users are not considered as valid credentials to connect MSK?