Authorization issue when connecting to MSK cluster

Question

I have configured an MSK cluster and allowed public access through SASL/SCRAM authentication method. Now I am facing an issue where I do not have the necessary permissions when using these credentials (specified in the Secrets Manager created with a custom key). The connecting client can perform certain operations but fails to fetch or create topics, nor publish a new message to the existing topic. I am using confluent and C# and here is an example of the configuration of my client.

```
BootstrapServers = Config.KafkaBootstrapServers,
SaslMechanism = SaslMechanism.ScramSha512,
SecurityProtocol = SecurityProtocol.SaslSsl,
SaslUsername = Config.Username, // username from secrets manager
SaslPassword = Config.Password, // password from secrets manager
ClientId = Config.Client,
Acks = Acks.All
```

How can I assign higher permissions? Since it is a managed Kafka service, there is no option to modify this on the broker level. And since there is no user behind these credentials, I cannot assign a specific policy to it. What are the options here?

Answer

The docs to configure your cluster to authenticate via IAM are available here.  https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html

You need to ensure that you deploy a policy with the kafka-cluster:CreateTopic permission to allow your clients to create a topic.

Best
Craig

Authorization issue when connecting to MSK cluster

Relevant content