AWS announces preview of AWS Interconnect - multicloud

AWS announces AWS Interconnect – multicloud (preview), providing simple, resilient, high-speed private connections to other cloud service providers. AWS Interconnect - multicloud is easy to configure and provides high-speed, resilient connectivity with dedicated bandwidth, enabling customers to interconnect AWS networking services such as AWS Transit Gateway, AWS Cloud WAN, and Amazon VPC to other cloud service providers with ease.

MSK Cluster connection failed with SASL authentication error for internal Kafka Users

Hi,

I have an MSK cluster with SASL/SCRAM enabled and created appropriate Super User for cluster with Secret to manage further users in cluster. If I create new users for example with Sarama client using this superuser as admin client, they appear in zookeeper config and kafka-configs, but connection for such users is failed with - Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512.

Can such internal Kafka users have access to MSK cluster with some additional configuration or every further user in MSK with SASL/SCRAM should have separate "Amazon_MSK_" prefixed secret and managed via KMS?

Ed Berezitsky
2 years ago
Error points to failed authentication. I think the user hasn't been created properly, or a client application doesn't define properties correctly. Please add more details, make sure the secret you are using is listed under MSK -https://docs.aws.amazon.com/cli/latest/reference/kafka/list-scram-secrets.html . Follow the documentation to properly define a new user for MSK: https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html
FilatovM
2 years ago
Let me clarify, I used only Kafka API and created user (No MSK secrets). In vanilla Kafka it works (all users are stored in ZK config), but in AWS such users fail authentication.

Can I use AWS admin user to create other users via default Kafka API? Or such users are not considered as valid credentials to connect MSK?

Topics: Analytics Security, Identity, & Compliance
Tags: AWS Identity and Access Management Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Language: English

FilatovM

asked 2 years ago735 views

1 Answer

Newest
Most votes
Most comments

Are these answers helpful? Upvote the correct answer to help the community benefit from your knowledge.

MSK supports SASL SCRAM with the users defined in AWS Secrets Manager ONLY. Please follow the documentation on how to create and associate a user with MSK. After that, use Kafka ACLs to manage permissions for that users.

Ed Berezitsky

answered 2 years ago

Relevant content

Lack of Super Users with MSK Cluster using SASL/SCRAM and ACLs
rePost-User-2637448
asked 3 years ago
MSK with SASL/SCRAM authentication. I can access the cluster but cant perform any action
Boris
asked 2 years ago
How to connect to public Amazon MSK cluster using SASL SCRAM using Confluent C# Kafka
amazon_msk_user
asked 2 years ago
MSK cluster with SCRAM-SHA-512 suddenly throws authentication error
rePost-User-0336140
asked 3 years ago
How do I troubleshoot authentication and permission issues when I use my Amazon MSK cluster with SASL/SCRAM authentication turned on?
AWS OFFICIALUpdated 3 months ago
How do I troubleshoot errors I receive when I try to connect to my Amazon MSK cluster?
AWS OFFICIALUpdated a year ago
How do I troubleshoot Lambda triggers that poll from Amazon MSK and self-managed Apache Kafka clusters?
AWS OFFICIALUpdated a year ago
How do I troubleshoot a cross-account AWS Glue job for an Amazon MSK cluster?
AWS OFFICIALUpdated a year ago
Create Kafka Client Keystore certificate for connecting AWS MSK with mTLS Authentication from AWS Glue
EXPERT
Nitin Kumar
published 5 months ago
Establishing Multi-tenant, Custom Domain-based Secure Connectivity to Amazon MSK through a Kafka Proxy
EXPERT
ThinkKolk@123
published a year ago

MSK Cluster connection failed with SASL authentication error for internal Kafka Users

Add your answer

Relevant content